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Overview 


While high school graduation rates are on the rise nationwide, too many students still never reach 
that milestone, with 7,000 on average dropping out every day. Recognizing that many students need 
additional support to succeed in school, Communities In Schools (CIS) works to provide and 
connect students with integrated support services to keep them on a path to graduation. CIS makes 
some services broadly available to all students at a school, and provides individual case management 
to those deemed most at risk of dropping out. 

This report presents the final implementation and impact findings from a two-year randomized 
controlled trial of CIS case management. This trial is half of a two-pronged national evaluation, the 
other half being a quasi-experimental study of the whole-school model. The report describes the 
implementation and effects of CIS case management in 24 mostly urban, low-income secondary 
schools in two states during the 2013-2014 school year. The study’s implementation research finds: 

• CIS site coordinators directly provide many services and also connect students with local 
partners and extant in-school services. While partner and in-school service providers help CIS 
meet students’ needs, it is challenging to monitor and evaluate their services’ quality. 

• During the second year of the study, approximately 80 percent of the students assigned to case 
management received services. These students received services an average of 20 times, for an 
average of just over 18 hours. Students classified as being at high risk of dropping out received 
services at a lower rate than moderate-risk students. 

• CIS case-managed students participated in support activities more frequently than non-case- 
managed students overall. 

The study also estimated the effect of case management on students’ nonacademic and more 
traditional school outcomes. After two years, the study found that case management had a positive 
effect on several nonacademic outcomes, including students’ attitudes about school and their 
relationships with adults and peers. However, the study also found that case management did not 
improve students’ school progress, achievement, attendance, or behavior. 

The final chapter of this report includes implications for practice based on the evaluation findings: 
ensuring that high-risk students receive the most attention; keeping students engaged with services; 
monitoring and adjusting services over time; working to ensure the quality of services from partner 
organizations; and emphasizing students in transition grades. CIS’s national office has already begun 
to implement changes based on the results of this evaluation. 

The report concludes by considering the results of this random assignment study of case manage¬ 
ment together with the results of the quasi-experimental analysis of the whole-school model. Taken 
together, the results suggest that whole-school models of integrated student support offer the 
promise of positive effects. The results also indicate, however, some areas to which support provid¬ 
ers may need to pay close attention, to ensure that students receive services that address their specific 
needs and that benefit them above and beyond the services already available. 
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Preface 


Although graduation rates have increased in recent years, the fact is that too many young people 
— about one million of them — drop out of school each year. To make it through high school, 
many students, especially those growing up in low-income communities, require academic and 
social support services that go beyond the classroom. While services may be available in the 
school and community, they are often scattered across numerous government agencies and 
nonprofit organizations, which makes it difficult to coordinate them and difficult for at-risk 
students to use them. Integrating these services is therefore viewed as a promising way to help 
students stay on track to graduate. 

Communities In Schools (CIS), a nonprofit organization with a national reach, offers a 
school-based model focused on organizing and supplementing the disparate services in a school 
and community. With a network of local affiliate offices in half the states in the nation (plus 
Washington, DC), CIS provides services of varying levels of intensity and duration based on 
students’ levels of need. Some CIS services are broadly available to all students at a school and 
others are directed at those most at risk of dropping out. This report is the second of two that 
present the findings from an experimental evaluation of the most intensive component of the 
CIS model — individual case management. This study found that the case management CIS 
provides succeeded in getting targeted students into more support activities and improving 
several of their nonacademic outcomes. However, the study found that these services did not 
have a positive effect on students’ attendance, academic performance, or behavior. 

CIS has committed itself to being a learning organization, regularly evaluating aspects 
of its program in order to improve its work on behalf of students. For CIS, external evaluation is 
not an endpoint or “check-the-box” activity intended solely to assess effectiveness, but rather an 
independent source of infomiation that can be used for program improvement. For example, 
after this evaluation found there was little differentiation in the type and amount of services 
provided to students receiving case management, CIS shifted from a two-tier to a three-tier 
model that specifies greater differentiation in services based on students’ needs. In addition, CIS 
relies heavily on services provided by partner organizations, and this evaluation found that 
monitoring the quality of these services can be challenging. In response, CIS has developed 
tools and additional guidance to help its network engage partners, with a focus on high-quality, 
evidence-based interventions. During the course of this evaluation, CIS also increased its own 
internal research capabilities, so that in the future it can both generate its own information and 
collaborate even more effectively with independent research organizations. 

Gordon L. Berlin 

President, MDRC 
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Executive Summary 


About one million students drop out of American high schools every year. 1 While graduation 
rates have increased over the last decade, one in five students still fail to graduate within four 
years. Compared with high school graduates, students who drop out are more likely to live in 
poverty, suffer from poor health, be involved in crime, and receive government assistance. 2 
Many students at risk of dropping out need academic and social services and other forms of 
support to make it through high school. However, services are often scattered across numerous 
government agencies and nonprofit organizations, which makes it difficult to coordinate them 
and difficult for at-risk students to use them. Integrating these services is therefore viewed as a 
promising way to help students stay on track to graduate. 3 

This report presents the findings from a random assignment study of individual case 
management provided by Communities In Schools (CIS), a national nonprofit organization 
focused on reducing dropout rates by providing integrated support to at-risk students in the 
nation’s poorest-performing schools. This study found that the case management CIS provides 
succeeded in getting targeted students into more support activities and improving several of 
their nonacademic outcomes. However, the study found that these services did not have a 
positive effect on students’ attendance, academic performance, or behavior after two years. 

What Is the CIS Model? 

Founded in 1977 by children’s advocate Bill Milliken, CIS works with K-12 schools in low- 
income communities that have many students at risk of failing or dropping out. CIS now serves 
nearly 1.5 million students and their families in 25 states and the District of Columbia. It is 
active in approximately 2,400 schools and 360 school districts. 4 

The ultimate goal of the CIS model is to reduce dropout rates by integrating community 
and school-based support services within schools. In the schools included in this evaluation, 


'National Center for Education Statistics, “Table 219.71: Population 16 to 24 Years Old and Number of 
16- to 24-Year-Old High School Dropouts (Status Dropouts), by Sex and Race/Ethnicity: 1970 through 2014” 
(website: https://nces.ed.gov/programs/digest/dl5/tables/dtl5_219.71.asp, 2015). 

2 Child Trends, High School Dropout Rates: Indicators on Children and Youth (Bethesda, MD: Child 
Trends, 2015). 

'Kristin Anderson Moore, Making the Grade: Assessing the Evidence for Integrated Student Supports 
(Bethesda, MD: Child Trends, 2014). 

Communities In Schools, 2014 Annual Report (Arlington, VA: Communities In Schools, 2014). 
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CIS works to accomplish this goal by providing what it calls “Level 1” and “Level 2” services. 5 
Level 1 services are broadly available to all students at a school and are usually short-term, low- 
intensity activities or services (for example, making clothing or school supplies available to 
students, organizing a school-wide career fair, or hosting a financial aid workshop for twelfth- 
graders). CIS’s internal standards say that a minimum of 75 percent of students at the school 
must be involved in at least one Level 1 service during the year. CIS site coordinators — those 
responsible for all CIS school-based operations — spend much of their time focused on more 
intensive Level 2 “case-managed” services, which they provide to a subset of students display¬ 
ing one or more significant risk factors for dropping out, such as poor academic perfomiance, a 
high absentee rate, or behavioral problems. In case management, site coordinators work with 
individual students to identify their needs, provide support directly and connect them with 
support in the school and community to address those needs, and regularly monitor their 
progress to ensure that their needs continue to be met. 

Level 2 Case Management 

This report focuses on the Level 2 case management component of the CIS model as 
implemented in middle and high schools. 6 The CIS model posits that providing individual 
support to at-risk students will provide them with the skills and resources they need to succeed. 

Through a review of data or because of a referral from an adult in the school, a site co¬ 
ordinator identifies a student as being at risk for eventually dropping out and seeks consent from 
a parent or guardian for the student to receive case management. The site coordinator assesses 
the student’s needs, develops an individual case plan, and sets goals with the student. Based on 
that plan, the site coordinator provides services appropriate to the student’s needs or connects 
the student to those services. The site coordinator monitors the student’s progress during the 
year and may adjust the plan based on changes in the student’s needs. 

The case management activities are expected to affect nonacademic outcomes related to 
students’ attitudes, behaviors, and relationships. The services provided to a student are intended 
to foster supportive relationships with adults and peers, encourage greater engagement with 
school, stimulate greater effort to meet academic and behavioral expectations, and increase the 
value that students see in their schooling. Effects on these nonacademic outcomes are hypothe¬ 
sized to indirectly affect more traditional school outcomes, such as attendance, performance in 


5 As discussed below, CIS shifted from a two-tier model of services to a three-tier model after this study 
was completed. 

6 The comprehensive CIS model is implemented in elementary, middle, and high schools. This evaluation 
of case management focuses only on secondary schools. 
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class, and disruptive behavior in school. These traditional school outcomes, in turn, are believed 
to predict students’ likelihood of eventually graduating from high school. 7 

How Was the CIS Model Evaluated? 

In its ongoing commitment to continuous improvement, the CIS national office looks to 
external organizations to provide independent and objective research intended to help its staff 
understand how its model is being implemented in schools and what its effect is on schools and 
students. A previous evaluation by ICF International suggested that young people who receive 
CIS services are more likely to achieve a number of positive outcomes than those who do not. 8 
As part of its participation in the federal Social Innovation Fund (SIF) grant program, CIS 
selected MDRC, a nonprofit, nonpartisan education and social policy research organization, to 
conduct an independent, two-study evaluation funded by the Edna McConnell Clark Founda¬ 
tion, with additional funding provided by The Wallace Foundation. 9 

One of the studies in this latest evaluation examines whether introducing the model 
with all of its elements improves school graduation rates, dropout rates, attendance rates, and 
state test scores. This component of the evaluation uses a quasi-experimental comparative 
interrupted time series design to estimate the effect of the whole-school model. 10 

This report addresses the second study in the evaluation, which focuses on CIS Level 2 
case management, examining service provision, student experiences, and student outcomes in 
the 2012-2013 and 2013-2014 school years. This study relies on a random assignment research 

7 Allensworth and Easton indicate that earning course credits and not failing core courses in ninth grade 
predicts eventual graduation. See Elaine M. Allensworth and John Q. Easton, The On-Track Indicator as a 
Predictor of High School Graduation (Chicago: University of Chicago, Consortium on Chicago School 
Research, 2005). Balfanz, Herzog, and Maclver have also found that as early as the sixth grade, 50 percent of 
future school dropouts in high-poverty schools show signs of falling off track: poor attendance, poor behavior, 
or poor course performance (that is, course failure). See Robert Balfanz, Liza Herzog, and Douglas Maclver, 
“Preventing Student Disengagement and Keeping Students on the Graduation Path in Urban Middle-Grades 
Schools: Early Identification and Effective Interventions,” Educational Psychologist 42, 4 (2007): 223-235. 

8 ICF International, CIS National Evaluation: Five Year Summary Report (Fairfax, VA: ICF International, 
2010); Allan Porowski and Aikaterini Passa, “The Effect of CIS on High School Dropout and Graduation 
Rates: Results From a Multiyear, School-Level Quasi-Experimental Study,” Journal of Education for Students 
Placed at Risk (JESPAR) 16,1 (2011): 24-37. 

9 The Edna McConnell Clark Foundation (EMCF) received a SIF grant from the federal Corporation for 
National and Community Service. CIS is a subgrantee to EMCF within the SIF program. Thus, while CIS was 
interested in continuing to evaluate its model, this evaluation is also being conducted as one of the required 
activities of the SIF grant program. It also aligns with EMCF’s interest in supporting organizations that are 
participating in evidence-generating research. 

10 For an explanation of this quasi-experimental design, see Marie-Andree Somers and Zeest Haider, A 
Tiered Approach to Dropout Prevention: A Quasi-Experimented Evaluation of Communities In Schools (New 
York: MDRC, 2017). 
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design. The middle schools and high schools that underwent random assignment for this study 
had more eligible students — those facing challenges that threatened to impede their progress 
toward graduation — than could be included on site coordinators’ caseloads. 11 These students 
were randomly assigned to join site coordinators’ caseloads (the case-managed group) or to 
continue with business as usual, with access to whatever other forms of support were available, 
including CIS Level 1 services (the non-case-managed group). Since random assignment 
created two comparable groups and the sample is large, students’ individual characteristics are, 
on average, the same in both groups, and any differences that emerge over time between these 
two groups can be attributed to CIS case management. The strength of the causal inference 
made possible by random assignment is why it is considered the gold standard of evaluation 
design. 

What Did the Study of CIS Level 2 Case Management Find? 

MDRC previously published an interim report after students had received case management for 
one year. 12 The current report focuses on 14 middle schools and 10 high schools in the second 
year of the study. In this second year the research team conducted additional implementation 
research and followed both the case-managed and non-case-managed students for another year 
of services. 13 The findings are consistent with the results found after one year. 

Two-Year Implementation Findings 

This report presents information about the CIS services available in study schools, pro¬ 
vides details about how CIS works in schools, and describes the amounts and types of support 
received by case-managed and non-case-managed students. Most schools in the study are 
located in or around large or midsize cities, serve predominantly black and Hispanic students, 
and serve low-income populations. 

Services Available in CIS Schools 

The study team learned that students in study schools have available to them a variety 
of support services from a variety of service providers. The services include Level 1 whole- 


1 'The quasi-experimental and random assignment components of the evaluation did not include any of the 
same schools. 

12 William Corrin, Leigh Parise, Oscar Cerna, Zeest Haider, and Marie-Andree Somers, Case Management 
for Students at Risk of Dropping Out: Implementation and Interim Impact Findings from the Communities In 
Schools Evaluation (New York: MDRC, 2015). 

13 Students in the comparison group continued not to receive case management during the second year of 
the study. One affiliate, with four study schools, opted not to continue in the study for the second year so that it 
could serve the students in the comparison group during that year. 
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school services provided by CIS site coordinators; school leaders reported that CIS is an 
important provider of many services, including mentoring, links to providers of basic necessi¬ 
ties, college and career preparation activities, and family-engagement activities. 

Working with Schools and Partners to Support Case-Managed Students 

CIS site coordinators provide many Level 2 services, and they also connect case- 
managed students with local partners and extant in-school services. When CIS staff members 
connect case-managed students to services provided by school staff members, those services 
“count” as Level 2 case-managed services. These services are often also available to non-case- 
managed students, which may mean that case-managed and non-case-managed students are 
sometimes receiving similar forms of support. The study team learned that one component of 
site coordinators’ roles is to act as advocates for their case-managed students, both by making 
sure students take advantage of services in their schools (which they might not do otherwise) 
and by checking with school staff members about students’ academic and behavioral progress. 

One of the goals of the second year of the study was to leam more about CIS partners’ 
role in supporting case-managed students. CIS’s partners provide many important services to 
case-managed students, especially mentoring, academic assistance, and family-engagement 
activities. However, some partners work more closely with site coordinators to plan services 
than others. It can be challenging in any case for CIS to monitor and evaluate the quality of the 
services partners provide, and that problem may be exacerbated when site coordinators and 
partners are not working together closely. 

Services Received by Case-Managed and Non-Case-Managed Students 

During the second year of the study, approximately 80 percent of the students in the 
CIS case-managed group received services. 14 Students received services as a result of case 
management an average of 20 times per year, totaling an average of just over 18 hours. One of 
the implementation findings from the first year of the study was that “high-risk” students 
received services as a result of case management about as many times and for about as many 
hours as lower-risk students. 15 In the second year, however, high-risk students received signifi¬ 
cantly different amounts of services than moderate-risk students. Specifically, 75 percent of 
high-risk case-managed students received services during the second year of the study, com¬ 
pared with 85 percent of moderate-risk students. This pattern held true for most of the catego- 

14 The 20 percent of students who did not receive services were still in the schools, as they were present to 
take the student survey in the spring of the second study year. 

15 “High-risk” students were those who in the year before the study began had failed a course, been chroni¬ 
cally absent, or been suspended. A student is considered chronically absent if he or she has an attendance rate 
below 90 percent. 
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lies of services provided to case-managed students: A greater proportion of moderate-risk 
students received services than did high-risk students. 

CIS case-managed students reported participating in more support activities than non¬ 
case-managed students, suggesting that CIS case management is providing something more 
than what students otherwise receive. Case-managed students reported higher levels of partici¬ 
pation in meetings with adults at school, meetings with mentors, tutoring, and career planning 
activities. For example, 39 percent of case-managed students participated in mentoring, com¬ 
pared with 27 percent of non-case-managed students; 75 percent of the case-managed students 
had one-on-one meetings with adults about academics, compared with 64 percent of non-case- 
managed students. Case-managed students also participated in support activities more times per 
year than non-case-managed students. 

Two-Year Impact Findings 

Level 2 case management is intended to advance CIS’s larger goal of improving stu¬ 
dents’ school progress. That is, CIS seeks to reduce the number of dropouts from a school and 
to increase its number of graduates. Because this study lasted only two years, it was not possible 
to track students through high school graduation. The study therefore examines nonacademic 
mediating outcomes and more traditional school outcomes. 

Nonacademic Outcomes 

These outcomes are behaviors and attitudes believed to precede students’ improved 
success in school. CIS case management had a positive and statistically significant effect on the 
rate at which students reported having a caring adult at home, at school, and outside of home 
and school, and on the quality of their peer relationships. Case management also had positive 
and statistically significant effects on students’ engagement with school, their educational 
attitudes, and their belief that education has value for their lives. However, there was no 
difference between case-managed and non-case-managed students in participation in school- 
and non-school-sponsored extracurricular activities, or in educational goals and expectations. 

Traditional School Outcomes 

CIS case management did not have an effect on students’ traditional school outcomes. 
Students in the case-managed and non-case-managed groups had similar rates of chronic 
absenteeism, attendance, core course failure, and credit accumulation, and similar course marks. 
However, case-managed students had more suspensions, on average, than non-case-managed 
students and this difference was statistically significant. Thus, it cannot be concluded that CIS 
case management improved students’ attendance, course performance, or behavior. 
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What Conclusions Can Be Drawn from MDRC’s Evaluation? 


While this study did not find that CIS case management had effects on students’ school out¬ 
comes, it did gather a great deal of information about CIS case management and the whole- 
school model. This information can help CIS improve case management practices and provide 
insight into how CIS might contribute to whole-school improvements. 

Considerations for CIS Case Management 

The evaluation findings suggest some areas where the CIS national office might con¬ 
sider providing more or different guidance and support to the affiliates and schools in its 
network, with the goal of affecting case-managed students’ school outcomes. They also offer 
some lessons for other integrated student-support organizations. The national office has already 
begun to implement some of these changes based on results from the evaluation. 

Ensuring that high-risk students receive the most attention. This study found that 
students potentially at higher risk for dropping out did not receive more services than those who 
may have been at less risk. After the first evaluation report, which included the finding that 
high-risk students did not receive meaningfully different services than moderate-risk students, 
CIS began moving beyond the two-level model implemented by the schools in this study. Its 
new three-tiered model is intended to provide more intensive support to the students with the 
greatest need. Further, CIS has begun classifying students based on their level of risk or need, 
and is focusing on providing more differentiated forms of support to meet students’ needs. 16 
CIS should continue to pay attention to this issue, and collect data to monitor whether the three¬ 
tiered model delivers more appropriate services to students with greater needs. 

Keeping students engaged. About 80 percent of the case-managed student sample re¬ 
ceived case management in the second study year. While keeping a large majority of students 
engaged for multiple years is a noteworthy accomplishment, it seems especially challenging to 
provide continuous services to high-risk students, as the percentage of those students who 
received case management declined substantially from the first to second year of the study. 
Other research has suggested that it may take at least two years for intensive case management 
to generate effects on students’ outcomes, 17 so CIS site coordinators and practitioners might 


16 These adjustments to the model occurred after this study, and thus this study’s results do not capture the 
effect of the changes. 

17 Maynard, Kjellstrand, and Thompson reported that the evaluations of the Check & Connect dropout 
prevention program that found positive effects on attendance and behavior were those that evaluated the 
program when implemented for at least two years. See Brandy R. Maynard, Elizabeth K. Kjellstrand, and 
Aaron M. Thompson, “Effects of Check and Connect on Attendance, Behavior, and Academics: A Random¬ 
ized Effectiveness Trial,” Research on Social Work Practice 24, 3 (2014): 296-309. 
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benefit from additional guidance about how to keep students engaged, particularly higher-risk 
students. 

Monitoring and adjusting services over time. During both years of the random as¬ 
signment study, there was variation in the extent to which site coordinators monitored the 
outcomes of case-managed students against their case plans, and revised those case plans and 
service provision to meet ongoing or changing student needs. Other researchers studying case 
management programs have emphasized the importance of carefully assessing students’ needs 
and meeting those needs with specific services. 18 One of the recent changes CIS has made is to 
expect a minimum number of check-ins per student, and to provide guidance to site coordina¬ 
tors about what should happen during the check-ins. CIS site coordinators and other staff 
members providing integrated support services may also benefit from guidance regarding how 
to monitor and adjust services, including the development of or training in systems intended to 
help them stay current about students’ needs. 

Working to ensure the quality of services from partner organizations. Finally, it 
can be inherently challenging for integrated student support organizations like CIS to monitor 
the quality of services provided by different organizations and individuals. Case-managed 
students in this study did receive more services than non-case-managed students, but they did 
not have different school outcomes. It may be that the services they received did not align with 
their most critical needs, or it may be that not all of the extra services they received were of high 
quality, in which case site coordinators could benefit from additional support to ensure their 
quality in the future. (Or both things could be true.) Site coordinators provide some services 
directly, but also rely heavily on school and community partners to provide others. CIS could 
help school-level and affiliate staff members make decisions about which partners and practices 
to bring into schools and about which in-school services to draw upon by helping them assess 
the existing evidence about the effectiveness of these partners or practices, and by identifying 
evidence-based practices or organizations that might be a fit in their schools. The CIS national 
office has already begun moving in this direction by developing tools for its network regarding 
partnership engagement, with a focus on high-quality, evidence-based providers. 

Working with students beginning in transition grades. Subgroup analyses for this 
study suggest that CIS case management may have the greatest potential to improve the 
outcomes of students who begin receiving it when they first enter middle or high school (that is, 
sixth- and ninth-graders). CIS site coordinators in secondary schools typically bring new 
students onto their caseloads when they are in those transition grades, and continuing this 
practice may help maximize the effects of case management on those students. 

ls Allen Schirm, Elizabeth Stuart, and Allison McKie, Tlie Quantum Opportunity Demonstration: Final 
Impacts (Princeton, NJ: Mathematica Policy Research, 2006). 
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Reflections Encompassing Both Studies 

This evaluation included two studies, as mentioned above, and it is important to consid¬ 
er what has been learned from both of them together. This study found that CIS case manage¬ 
ment succeeded in getting secondary school students into more support activities and improving 
several of their nonacademic outcomes, but that it did not have a positive effect on their more 
traditional school outcomes. Meanwhile, the quasi-experimental study examined the CIS model 
as a whole in elementary, middle, and high schools. That whole-school study found that after 
three years of implementation, on-time graduation rates improved by a statistically significant 
amount in the CIS high schools, relative to what would have been expected given those schools’ 
baseline trends. Graduation rates also improved in the comparison schools used in the study, 
and it is unclear whether the CIS model was more effective than the strategies used by the 
comparison schools. However, the findings do suggest that the model may be at least as 
effective as these other approaches. At the middle school level, ELA test scores did not improve 
in schools implementing the CIS model, whereas they did significantly improve for a group of 
similar comparison middle schools. At the elementary school level, attendance rates improved 
in schools implementing the CIS model more than they improved in a group of similar compari¬ 
son schools. 

What sense can be made of the two studies’ findings taken together? While the whole- 
school study suggested that schools implementing the CIS model experienced improved 
graduation and dropout rates — and that CIS may be at least as effective as other approaches — 
the study of case management found that case management does not improve student outcomes 
thought to predict graduation within two years, though it does get students into more support 
activities and improves their nonacademic outcomes. 

There are several possible explanations for this combination of findings. It may be that 
having a CIS site coordinator who works closely with a group of case-managed students allows 
other support staff members, such as guidance counselors and social workers, to work more 
with the non-case-managed students than they would have otherwise. If that were true, the 
school as a whole might improve because of CIS’s presence, even though the students receiving 
case management did not improve more than students randomly assigned to the non-case- 
managed group. The non-case-managed students would then have benefited indirectly from the 
presence of a CIS site coordinator practicing case management. 

A second possibility is that the Level 1 services CIS provides, which are accessible to 
the majority of students in a school, may change school-wide outcomes more than case man¬ 
agement can, with its focus on a small minority of targeted students. This notion is supported by 
school leaders’ reports that CIS is an important part of their schools and that CIS is an essential 
provider of support services. 
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Finally, while previous research indicates that attendance, behavior, and course perfor¬ 
mance are correlated with graduation, it may be that this study was not able to track case- 
managed students for long enough to see effects on these outcomes, or that these outcomes 
might not be the only factors that predict graduation. It is possible that over time, the positive 
effects of case management on students’ nonacademic outcomes would translate into positive 
effects on their more traditional school outcomes. As indicated above, previous research 
suggests that case management can take at least two years to affect students’ attendance and 
behavior. Alternatively, it may be that those nonacademic improvements could end up making a 
difference in keeping students in school, even though they do not alter the traditional school 
outcomes tracked in this study. 

It remains uncertain, however, whether CIS’s model makes a bigger difference than al¬ 
ternative approaches to school improvement. A random assignment study of the whole-school 
model may be the next step that would provide the most information about the model’s effect 
relative to that of other programs and strategies. If an evaluation included a cost study on the 
implementation of the CIS model, along with cost data on the strategies and interventions used 
by the control schools, it could also determine the CIS model’s relative cost-effectiveness, 
which would ultimately provide the most useful decision-making information for school and 
district leaders. 

As CIS and other integrated student support organizations continue to work toward ad¬ 
dressing students’ needs, they can leam from both completed and ongoing research to refine 
their models. While graduation rates have risen in the last decade, it remains the case that far too 
many students drop out of school — roughly 7,000 every day. 19 This evaluation suggests that 
whole-school models of integrated student support services do offer the promise of positive 
effects. However, in the actual implementation of tiered-support models, it appears to be 
important to pay close attention to how that tiered support might improve conditions for 
students above and beyond the kinds of support already available in a school. As CIS and other 
similar organizations continue to refine their models, they should pay particular attention to 
reaching the students who are most in need of support and to connecting students with high- 
quality services. In addition, they may want to target schools for CIS implementation that do not 
already provide a broad range of services for students, or where service provision is particularly 
fragmented, as these may be the schools where they would have the most to offer. 


19 National Center for Education Statistics (2015). 
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Chapter 1 

Introduction 


Every day more than 7,000 students drop out of school. 1 Among black and Hispanic boys, the 
dropout rates are 42 percent and 48 percent, respectively. 2 Even though high school graduation 
rates have risen this century, too many students who enter public high school (one in five) do 
not graduate within four years. 3 Compared with high school graduates, dropouts are more likely 
to live in poverty, suffer from poor health, be involved in crime, and be dependent on social 
services. 4 

Students at risk of dropping out often need academic and social services and other 
forms of support to make it through high school to graduation. The support services available to 
address these students’ needs are scattered across government agencies and nonprofit organiza¬ 
tions in communities with poorly performing schools, which often means that students either do 
not make use of the services available or receive specific services in isolation. This lack of 
coordination limits the potential of any service to change the trajectory of a student who is 
headed toward dropping out. Within schools, teachers and principals are often overwhelmed by 
the emotional, social, and personal issues facing students. Integrating student support services 
(that is, connecting community agencies and organizations with schools and coordinating the 
services provided to students) is viewed as a promising way to provide assistance to school staff 
members and help students stay on track to graduate. 5 


The Communities In Schools Model 

Communities In Schools (CIS) provides an integrated student support model to schools and 
communities. Founded in 1977 by children’s advocate Bill Milliken, CIS works with low- 
income students at risk of failing or dropping out of the nation’s poorest-performing schools. 
The organization has an extensive national reach and now serves more than 1.3 million students 
and their families. 6 It is active in nearly 2,500 schools, and the national office oversees 17 state 
offices as well as a network of over 160 local affiliates in 25 states and the District of Colum- 


'National Center for Education Statistics (2015). 
2 Schott Foundation for Public Education (2012). 
3 Murnane (2013); Stetser and Stillwell (2014). 
4 Child Trends (2014). 

5 Moore (2014). 

Communities In Schools (2015). 
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bia. 7 The national office is responsible for developing and enhancing the CIS model; communi¬ 
cating with national audiences and advocating for educational reform that includes integrated 
student support services; fostering collaboration within its network; supporting research about 
and evaluation of the model; and establishing national partnerships intended to generate 
resources and funding for members of the network. The local affiliates — each of which is an 
independent organization with its own board of directors — oversee the implementation of the 
model in schools. They also build community partnerships and develop local funding and 
resources to support the program. A CIS site coordinator implements Level 1 and Level 2 
services at a school, sometimes assisted by additional CIS staff members, interns, or Ameri- 
Corps members. The site coordinator engages in yearly school-level needs assessments and 
planning with school leaders, teachers, and other members of the staff. 

CIS has created a comprehensive service model that seeks to reduce dropout rates by in¬ 
tegrating community-based support within schools through both preventive “Level 1” services, 
which are available to many or all students in the school, and intensive, targeted, and sustained 
“Level 2” case management services for students who are displaying one or more significant 
risk factors for dropping out, such as poor academic performance, a high absentee rate, or 
behavioral problems. 8 The differences between the two service levels can be described in terms 
of accessibility and duration or intensity. Level 1 services are broadly accessible and usually 
consist of short-term, low-intensity activities or assistance that students usually pursue voluntar¬ 
ily (for example, making clothing or school supplies available to students and hosting school¬ 
wide events). 9 They also include short-term “crisis” interventions when an extreme event 
disrupts a student’s life (for example, finding a solution if the power is turned off at the stu¬ 
dent’s home or providing short-term counseling in response to a traumatic event). 

The current study and this report focus on the implementation and effects of Level 2 
case management services. These services target specific students, typically over a longer term 
and at a higher intensity than Level 1 services. Figure 1.1 is a representation of CIS’s case 
management logic model. The “Context/Resources” column in the figure shows the kinds of 
resources and contextual factors that support or influence the case management work. For 
example, the financial resources available and the number of students in a school influence how 


’Communities In Schools (2015). 

8 As discussed in Chapter 5, CIS shifted from a two-tier to a three-tier model of services after this study 
was completed. 

9 While Level 1 services are school-wide, some are not truly available to all students because they target a 
broad subgroup, such as a college financial aid application workshop open only to twelfth-graders. Also, 
although discussed here primarily in terms of student services, Level 1 may also include services for parents. 
Such services are intended to strengthen students’ support at home and to more fully engage parents in order to 
improve a school’s overall climate. 
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Figure 1.1 


CIS Case Management Logic Model 
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many site coordinators might be assigned there. And site coordinators look to the community’s 
existing organizations that provide youth services (such as counseling, recreation, or jobs) to 
build partnerships and foster opportunities for students. 

The second column, labeled “Activities,” outlines the steps that are part of the case 
management process. Through a review of data or because of a referral from another adult in 
the school, the site coordinator identifies a student as being at risk of eventually dropping out 
and begins the process of adding the student to his or her caseload, first seeking consent from 
the parent(s) or legal guardian(s). The site coordinator assesses the student’s needs, develops an 
individual case plan, and sets goals with the student, then provides the appropriate services or 
connects the student to other individuals or organizations that can provide them. As discussed in 
this report, many of the services students receive as a result of case management may be 
services that already exist in the school (for example, tutoring offered by a teacher), but that the 
site coordinator ensures the case-managed student takes advantage of. During the year, the site 
coordinator monitors the student’s progress and adjusts the plan as necessary as the student’s 
needs change. 

Case management activities are expected to affect nonacademic “Mediating Factors” 
(the third column in Figure 1.1) related to students’ attitudes, behaviors, and relationships. The 
services provided to a case-managed student are intended to foster supportive relationships with 
peers and adults (including those adults responsible for coordinating the services), encourage 
greater engagement with school, stimulate greater effort to meet academic and behavioral 
expectations, and increase the value that the student sees in his or her schooling. Effects on 
these nonacademic mediating factors are eventually expected to lead to changes in traditional 
student perfomiance measures such as attendance, credit accumulation, and behavior in school 
that requires a disciplinary response, as listed in the final column of Figure 1.1, “Outcomes.” 


Evaluating the CIS Model 

In its ongoing commitment to continuous improvement, the CIS national office looks to 
external organizations to provide independent and objective research intended to help its staff 
understand how its model is being implemented in schools and how it is affecting schools and 
students. The national office, with funding from Atlantic Philanthropies, previously commis¬ 
sioned ICF International to conduct a five-year evaluation of its comprehensive model and the 
case management component. 10 The findings from that evaluation suggested that young people 
who receive CIS services are more likely to achieve a number of positive outcomes than those 


10 ICF International (2008); ICF International (2010); Porowski and Passa (2011). 
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who do not. 11 Seeking to strengthen this evidence base, CIS welcomed further external evalua¬ 
tion of its comprehensive model and the case management component. 12 Accordingly, MDRC, 
a nonprofit, nonpartisan education and social policy research organization, is conducting an 
independent evaluation of CIS. This evaluation is supported primarily by the federal Social 
Innovation Fund. (See Box 1.1 for details.) 

MDRC’s evaluation of CIS includes two separate studies. One study evaluates the ef¬ 
fect of CIS’s “whole-school model” of integrated student support services, including both Level 
1 and Level 2 services, using a quasi-experimental, comparative interrupted time series design. 
In this design, the program’s effects are evaluated by determining whether schools that imple¬ 
mented an intervention (in this case, the CIS model) “deviated” from their baseline trends by a 
greater amount than a group of similar comparison schools. The current report focuses on the 
second component of the evaluation: a rigorous student-level randomized controlled trial of the 
implementation and effect of Level 2 case management specifically. 

The Study of the CIS Whole-School Model 

The quasi-experimental evaluation of the whole-school model estimates the CIS mod¬ 
el’s effect on several student outcomes: graduation and dropout rates in high schools, and 
attendance rates and performance on state tests in elementary, middle, and high schools. The 
schools in the study are located in Texas and North Carolina, two states where CIS has a strong 
presence. This component of the evaluation examined the effect of the CIS model in the first 
three years it was implemented. 

The study found that on-time graduation rates increased — and dropout rates decreased 
— in the study schools after the CIS model was launched. It is not clear whether these im¬ 
provements were caused by the CIS model, however, because the comparison schools used in 
the study may not accurately represent what would have happened to the CIS schools had they 
not implemented the CIS model. The comparison schools’ baseline graduation rates were 
almost 10 percentage points higher than those of the CIS schools. Although the size of this 
difference is acceptable based on commonly used criteria for baseline equivalence, the differ¬ 
ence could be large enough to affect schools’ incentives and decisions about their programs. 


1 Porowski and Passa (2011). 

12 The Edna McConnell Clark Foundation (EMCF) received a Social Innovation Fund (SIF) grant from the 
federal Corporation for National and Community Service. CIS is a subgrantee to EMCF within the SIF 
program. While CIS was interested in continuing to evaluate its model, this evaluation is being conducted as 
one of the required activities of the SIF grant program. It also aligns with EMCF’s interest in supporting 
organizations that are participating in evidence-generating research. This evaluation is also supported by funds 
from The Wallace Foundation. 
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Box 1.1 

The EMCF Social Innovation Fund 


The Social Innovation Fund (SIF), an initiative enacted under the Edward M. Kennedy Serve 
America Act, directs millions of dollars in public-private funds to expand effective solutions in 
three issue areas: economic opportunity, healthy futures, and youth development and school 
support. This work seeks to create a catalog of proven approaches that can be replicated in 
communities across the country. The SIF generates a 3:1 private-public match, sets a high 
standard for evidence, empowers communities to identify solutions to social problems, and 
creates an incentive for grant-making organizations to direct funding more effectively to 
promising programs. Administered by the federal Corporation for National and Community 
Service (CNCS), the SIF is part of the government’s broader agenda to redefine how evidence, 
innovation, service, and public-private cooperation can be used to tackle urgent social chal¬ 
lenges. 

The Edna McConnell Clark Foundation (EMCF), in collaboration with MDRC and The 
Bridgespan Group, is leading a SIF project that aims to expand the pool of organizations with 
proven programs that can help low-income young people make the transition to productive 
adulthood. The project is particularly focused on young people who are at the greatest risk of 
failing or dropping out of school or of not finding work, who are involved or likely to become 
involved in the foster care or juvenile justice system, or who are engaging in risky behavior 
such as criminal activity or teenage pregnancy. 

EMCF, with its partners MDRC and Bridgespan, selected an initial group of nine programs 
and a second group of three programs to receive SIF grants: BELL (Building Educated Lead¬ 
ers for Life), the Center for Employment Opportunities, Children’s Aid Society-Carrera 
Adolescent Pregnancy Prevention Program, Children’s Home Society of North Carolina, 
Communities In Schools, the Gateway to College Network, PACE Center for Girls, Reading 
Partners, The SEED Foundation, WINGS for Kids, Youth Guidance, and Children’s Institute, 
Inc. These organizations were selected through a competitive selection process based on 
evidence of impacts on economically disadvantaged young people; a track record of serving 
young people in communities of need; strong leadership and a potential for growth; and the 
financial and operational capabilities necessary to expand to a large scale. 

The EMCF Social Innovation Fund initiative, called the True North Fund, includes support 
from CNCS and 15 private coinvestors: EMCF, The Annie E. Casey Foundation, The Duke 
Endowment, The William and Flora Hewlett Foundation, The JPB Foundation, George Kaiser 
Family Foundation, The Kresge Foundation, Open Society Foundations, The Penzance 
Foundation, The Samberg Family Foundation, The Charles and Lynn Schusterman Family 
Foundation, The Starr Foundation, Tipping Point Community, The Wallace Foundation, and 
the Weingart Foundation. 
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In the study’s middle schools, the CIS model appears to have been less successful at 
improving students’ scores on state tests than the strategies or refonns used by the comparison 
schools. The model had no effect on state test scores in elementary schools or high schools. In 
elementary schools, the CIS model appears to have had a positive and statistically significant 
effect on attendance rates equivalent to about one extra day of school during the year. The 
model had no effect on attendance in middle schools or high schools. 

The Study of Level 2 Case Management 

As described above, a central component of the CIS whole-school model is Level 2 
case management for a subset of students identified as being at risk of not making satisfactory 
progress in school and of dropping out. The second part of MDRC’s evaluation examines the 
effect of CIS’s Level 2 case management on student outcomes, particularly progress toward 
graduation, and explores how case management is implemented. 

To accomplish these goals, this study relies on a random assignment research design, 
often referred to as the gold standard of evaluation design. Random assigmnent is a lottery-like 
process in which individuals are assigned either to participate in a specific program or else to 
continue with the “business-as-usual” altemative(s). In cases where there are more individuals 
interested in and eligible for a program than there are available slots, this process provides a fair 
way to detennine who participates in the program and also creates conditions for two groups 
that are equivalent — that is, the characteristics of the group assigned to participate in the 
program should be the same as those of the group assigned to continue with business as usual. 
By comparing the outcomes of these groups, it is possible to assess the effect of the program 
rigorously, because any differences between them after random assigmnent can be attributed to 
the program. Box 1.2 includes details about student recruitment and random assigmnent, which 
took place during the first year of the study. 

Meanwhile, MDRC’s implementation research examined the details of the case man¬ 
agement process, shedding light on the impact findings and aiming to provide useful infor¬ 
mation to CIS about how it might improve this component of its model. 

This study of CIS case management was conducted in 28 CIS secondary schools in 5 
affiliates. In each of the study schools, eligible students were randomly assigned to two groups. 
Students in the “case-managed group” were offered the opportunity to receive CIS case man¬ 
agement and students in the “non-case-managed group” were not, although they did retain 
access to whatever other services were available in their schools (including Level 1 CIS 
services). This design makes it possible to evaluate the effect of CIS Level 2 case management 
over and above the effect of the Level 1 and other services provided in the study schools. 
Students in the case-managed and non-case-managed groups were followed for one school year 
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Box 1.2 


Recruitment and Random Assignment 

After schools were selected for the study, the research team worked with CIS affiliates and site 
coordinators to plan for student recruitment. As discussed in Chapter 2, students are typically 
identified for CIS case management through referrals and recommendations from teachers, 
administrators, guidance counselors, or other student support staff members on campus. This 
identification process often means that CIS site coordinators do not fill their caseloads until the 
second semester. In order to enroll students in case management early in the school year, the 
research team helped affiliates work with their local school districts to collect data to preidenti¬ 
fy students who would be eligible, based on issues such as course failures, low grade point 
averages, poor attendance, or behavioral infractions. While using school records was standard 
practice for a few study schools, whose district provides CIS with a list of at-risk students each 
year, it was new for most schools and affiliates in the study. Students were also considered to 
be eligible for case management if they were identified through one of the more typical 
pathways — being referred to CIS by a school staff member or parent due to a recognized 
need or being identified directly by a CIS site coordinator. 

As part of the student recruitment process, CIS affiliate and school-based staff members 
provided eligible students with information about CIS and the study. At each school, students 
who returned both the CIS and study consent forms were entered into MDRC’s random 
assignment database.* Students were included in the random assignment database whether or 
not their parents consented to study participation, as service receipt was not contingent on 
consent to enter the study. With the use of random assignment, every student eligible for case 
management (that is, every student with consent to participate and a demonstrated need for 
support) at each school had the same chance to be served. MDRC conducted random assign¬ 
ment lotteries for each school to determine which students would fill the available CIS case 
management slots (the case-managed group) and which students would constitute a compari¬ 
son group and continue with the standard services and other forms of support available to them 
at their schools (the non-case-managed group). 1 Although this random assignment process was 
different from the typical CIS recruitment process, involving additional effort at the start of the 
year, in interviews site coordinators generally reported that the students who were randomly 
assigned to their caseloads were similar to the non-study students on their caseloads and to 
students they had served in previous years. Further, most site coordinators said that their case 
management processes and activities during the study years were similar to those in the 
preceding years. 


*These forms had to be signed by a parent or legal guardian unless the student was already an adult 
(18 years old). 

In some schools, two lotteries were performed, with the first occurring after a certain number of stu¬ 
dents had been recruited but before the school’s overall study recruitment target had been reached. Doing 
so allowed CIS site coordinators to begin providing case management to the students in the first lottery 
sooner. A second lottery occurred at 7 of the 28 schools in the study. 





in all of the study schools. In the second year of the study, 24 of the 28 study schools (in four of 
five affiliates) agreed to continue to participate. Students in the case-managed group in these 
schools could continue to receive case management for a second year (and students in the non¬ 
case-managed group were not offered case management). 13 In this group of 24 schools, both 
case-managed and non-case-managed students were followed for two school years. 

Interim Report Findings 

The interim report from this study focused on the implementation of CIS case manage¬ 
ment and its effects after one year of services in all study schools. The main implementation 
finding was that each step of the case management process — identification, assessment, case 
planning, service provision, and monitoring and adjusting — was carried out in the study 
schools, but that there was some variation in how these steps were carried out. 

During the first year of the study, case-managed students participated in an average of 
2.5 activities per month that were the result of case management, for a total of 16 hours of 
services. The research team found that “high-risk” case-managed students — those who had 
failed a course or who had been chronically absent or suspended in the previous year — did not 
receive more of these services than others. 14 

The first report also included the finding that after one year, case management had posi¬ 
tive effects on students’ reports of having caring, supportive relationships with adults outside of 
home and school; on the quality of their friendships; and on their belief that education matters 
for their future. However, there was no evidence that CIS case management had improved 
students’ attendance or course performance, or reduced behaviors that lead to disciplinary action 
— outcomes that would have suggested they had improved their chances of graduation. 

The present report describes the effect of CIS case management after two years in the 
24 schools that participated for a second year. 15 It also provides additional information on the 

13 One affiliate, with four study schools, opted not to continue in the study for the second year so that it 
could serve the students in the comparison group during that year. While it is common practice for CIS case- 
managed students to receive multiple years of service, participating in this study may have led some site 
coordinators to increase their efforts to keep students on their caseloads. While this study did not explicitly 
measure whether or not they did so, site coordinators in locations that require annual parental consent for 
students to receive CIS services (for example, Texas) probably made somewhat greater efforts to get new 
consent forms from parents during the second year of the study so they could continue to serve as many 
students as possible. 

,4 A student is considered chronically absent if he or she has an attendance rate below 90 percent. 

15 Because the findings in this report are based on a subgroup of the study schools, the results included here 
are not directly comparable to those in the first report. The appendixes of the present report include implemen¬ 
tation and impact findings after one year of services for the subgroup of schools that continued to participate in 
the second year of the study. 
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implementation of case management in these schools, including supplemental details about the 
case management process, CIS’s collaboration with partners in the school and community, and 
the role of school-based staff members (for example, guidance counselors and social workers) 
in providing support to students and working with CIS. 

Research Questions 

The main impact research questions for the second year of the random assignment 
study are similar to the questions for the first year: 

• During the second year of the study, does case management reduce the num¬ 
ber of students who are chronically absent? 

• During the second year of the study, does case management reduce the num¬ 
ber of students who fail one or more of their core academic classes? 

Some of the implementation research questions during this second year of the study 
were also similar to those asked during the first year: 

• During the second year of the study, what services do students receive, how 
many times do they receive them, and for how long do they receive them? 

• During the second year of the study, does case management create a differ¬ 
ence between the experiences of case-managed students and the experiences 
of non-case-managed students? 

• What are the circumstances under which case management is implemented? 

But the first year of implementation research also raised some additional questions that 
this second year of the study explores further: 

• As case management is implemented, what other services are available in 
schools, what roles do school staff members play in supporting students, and 
how does CIS fit into that context? 

• How does CIS work with partners and school staff members to provide case 
management services to students? 

Data Sources 

To address these research questions, the study draws upon a number of quantitative and 
qualitative data sources. The data from these sources provide information that corresponds to 
the categories presented in the case management logic model: context, activities, mediating 
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factors, and outcomes (Figure 1.1). Additional details about the following data-collection 
activities are available in Appendixes D and E (including the full text of surveys administered 
during the evaluation). 

Adult surveys. The study team administered a survey to school leaders and CIS site 
coordinators at the participating schools in the spring and summer of 2013 (the end of the first 
year of the study) and another in the spring and summer of 2014 (the end of the second year of 
the study). 16 The surveys provide infonnation about school and community context; what kinds 
of support programs and services were available to case-managed and non-case-managed 
students; and what issues the respondents saw as important to the school. (See “Context” and 
“Activities” in Figure 1.1.) 

In-person interviews. The research team conducted implementation research in a sub¬ 
set of study schools in both years of the study. In the spring of 2014, the study team visited 14 
of the 24 schools participating in the evaluation across all four local affiliates who continued in 
the second year of the study. 17 The main focus of these visits was to conduct in-person inter¬ 
views with CIS site coordinators, school principals, school support staff members, case- 
managed and non-case-managed students, CIS partners, and CIS affiliate staff members. The 
data collected from all of these interviews provide infonnation about the local implementation 
contexts and program implementation. (See “Context” and “Activities” in Figure 1.1.) 

Management information system data. A management infonnation system is a data¬ 
base that holds infomiation on program operations and that can produce reports on a program’s 
management. In this study, management infonnation system data recorded by CIS site coordi¬ 
nators are used to measure the amount and types of services case-managed students received. 
Depending on the local affiliate, these data came from CIS’s national management infonnation 
system or from similar state or local systems. (See “Activities” in Figure 1.1.) 

Student surveys. Students in both study groups — case-managed and non-case- 
managed — participated in a baseline survey in the fall of 2012, and follow-up surveys in the 
spring of 2013 (the first year of the study) and the spring of2014 (the second year of the study). 
On the follow-up surveys, students reported what kinds of support they received in and out of 
school, allowing the study team to compare support services received by case-managed and 
non-case-managed students. In addition, these surveys provide information about nonacademic 


16 In a majority of the study schools, the school leader survey was completed by the principal or assistant 
principal, but some cases it was completed by another staff member identified by the principal as being 
knowledgeable about student support services at the school (for example, the head of guidance, a guidance 
counselor, or a social worker). 

17 All of these schools were also visited during the first year of the study. 
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outcomes such as students’ engagement with school, their relationships with peers and adults, 
and their educational aspirations. (See “Mediating Factors” in Figure 1.1.) 18 

Student records data. To answer the research questions about the effect of case man¬ 
agement on students’ school outcomes, the study relies on student records data obtained from 
the participating school districts. These districts provided baseline data (before random assign¬ 
ment) about students for the 2011-2012 school year and follow-up data (after random assign¬ 
ment) for the 2012-2013 and 2013-2014 school years (after case-managed students could have 
received one and two years of service, respectively). The baseline data make it possible to 
describe the sample of students in the study and analyze whether the case-managed students 
were comparable to their non-case-managed peers at the start of the study. 19 The follow-up data 
provide the information necessary to determine whether case management had an effect on 
measures related to attendance (such as chronic absenteeism and attendance rates), course 
perfonnance (such as course failure rates, average grades, and credits earned in core courses), 
and behavior (such as number of suspensions). (See “Outcomes” in Figure 1.1.) 


Characteristics of Affiliates and Schools in the Evaluation 

As described in the first report, the MDRC research team worked with the CIS national office 
during the 2011-2012 school year to recruit affiliates and schools. The research team selected 
affiliates with multiple schools in which CIS believed there were at least twice as many students 
in need of Level 2 case management as could be served. It was important that random assign¬ 
ment only took place in schools where CIS lacked the resources to serve all the students in 
need: When need exceeds available resources, random assignment is a fair way to allocate those 
scarce resources. Twenty-eight schools — 16 middle schools and 12 high schools — across the 
five affiliates were selected to participate in the study during the first year. Twenty-four schools 
— 14 middle schools and 10 high schools — in four CIS affiliates continued in the study for a 
second year. Two of the affiliates that continued in the study for a second year were in North 
Carolina and two were in Texas. Two affiliates were located in large cities and two were in 
midsize cities, though some of the schools they serve are located in nearby suburban and rural 
communities. 


1 ^ During the first year of the study, students responded to baseline surveys before case management be¬ 
gan. Both the baseline and follow-up surveys make use of items from the California Healthy Kids Middle 
School Survey (specifically items from Module A and resilience items from Module B) with permission of the 
California Department of Education and WestEd. 

19 In addition, because a prior measure is a good predictor of a later outcome, the baseline data are included 
as co variates in the impact analyses, helping to improve the precision of the impact estimates. 
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Table 1.1 shows the characteristics of the study schools participating in the second year 
of the study, the averages of all schools in the study states (Texas and North Carolina), and the 
national averages for middle and high schools. A majority of the schools in the study are located 
in or around large or midsize cities, serve predominantly black and Hispanic students, and serve 
relatively low-income populations. All schools in the study receive Title I funds and 74 percent 
of the high school students and 89 percent of the middle school students were eligible to receive 
free or reduced-price lunches. 20 In addition, the study schools are relatively large, with high 
schools serving an average of approximately 1,700 students and middle schools serving an 
average of approximately 780. Compared with other schools in the study states and the nation, 
the study schools were larger, more urban, served more low-income students, and served more 
black and Hispanic students and fewer white students. Study high schools also had higher 
proportions of ninth-graders and lower proportions of twelfth-graders than other high schools in 
the study states and the nation, a sign of higher dropout rates. 

Reports of Student Needs and Services Available in the Study Schools 

In addition to collecting publicly available data about the study schools, the research 
team collected information from school leaders and CIS site coordinators on the challenges 
students in their schools faced in the second year of the study. As detailed in Table 1.2, school 
leaders and site coordinators indicated that poor academic perfomiance and lack of parental 
involvement and support affected the greatest number of students in their schools — nearly half 
or more. In addition, they estimated that between a quarter and a third of students faced chal¬ 
lenges related to poor attendance, behavior or discipline problems, high-risk social behavior (for 
example, drug use or gang participation), or family instability. At least 80 percent of school 
leaders and site coordinators indicated that poor academic perfomiance, poor attendance, 
behavior or discipline problems, and high-risk social behavior were a high priority for their 
schools and for CIS. Finally, as shown in the final column of Table 1.2, school leaders’ and site 
coordinators’ estimates of the percentage of students at risk of dropping out were 24 percent and 
36 percent, respectively. 21 

School leaders and site coordinators also provided information about the support ser¬ 
vices available to address the needs of the students in their schools — a critical factor for 
understanding the study schools’ environments and the kinds of services that are available to 
both case-managed and non-case-managed students. Overall, the infonnation shared by school 


‘"Title I is the federal funding stream designated for schools serving low-income students. 

21 On every one of the seven issues in the survey, site coordinators estimated a larger percentage of stu¬ 
dents affected than did school leaders. 
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Table 1.1 


Selected Characteristics of the Study Schools, 
Schools in Study States, and the 
National Population of High Schools and Middle Schools 


Characteristic 

Study Schools in 

Schools Study States 3 

National 

Schools 3 

High schools 

School locale (%) 

Large or midsize city 

80.0 

25.0 

19.2 

Urban fringe or large town 

20.0 

23.0 

34.1 

Small town or rural area 

0.0 

52.0 

46.6 

Schools with Title 1 status (%) 

100.0 

75.7 

55.0 

Average school enrollment 

1,696.9 

901.7 

790.5 

Average student enrollment (%) 

Grade 9 

30.4 

27.1 

24.9 

Grade 10 

25.2 

25.0 

24.5 

Grade 11 

22.7 

24.4 

24.3 

Grade 12 

21.7 

23.4 

25.9 

Students receiving free or reduced- 

price lunch (%) 

74.2 

53.1 

47.6 

Race/ethnicity of students (%) 

Black 

36.7 

16.1 

15.1 

Hispanic 

41.5 

36.4 

20.8 

White 

16.6 

42.6 

56.2 

Other 

4.6 

4.9 

8.0 

Female students (%) 

48.2 

48.9 

48.1 

Average number of full-time teachers 

93.7 

60.6 

49.6 

Number of schools 

10 

1,784 

16,312 


(continued) 


leaders and site coordinators suggests that the study schools offer a wide variety of services 
meant to address students’ needs. 22 While Chapter 2 will focus more specifically on the services 
made available via CIS, Table 1.3 details the types of support services school leaders reported 


22 School leader and CIS site coordinator surveys were the primary data sources for this information, as 
such data are not available consistently from the school districts participating in the study. 
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Table 1.1 (continued) 


Characteristic 

Study 

Schools 

Schools in 
Study States 3 

National 

Schools 3 

Middle schools 

School locale (%) 

Large or midsize city 

91.7 

30.5 

19.9 

Urban fringe or large town 

0.0 

27.7 

41.8 

Small town or rural area 

8.3 

41.8 

38.2 

Schools with Title 1 status (%) 

100.0 

82.2 

69.6 

Average school enrollment 

780.0 

673.9 

628.9 

Average student enrollment (%) 

Grade 6 

32.5 

33.3 

32.6 

Grade 7 

33.2 

33.6 

33.7 

Grade 8 

31.4 

33.1 

33.5 

Students receiving free or reduced- 

price lunch (%) 

88.4 

60.3 

52.5 

Race/ethnicity of students (%) 

Black 

40.4 

16.6 

16.1 

Hispanic 

49.0 

39.4 

21.8 

White 

6.5 

38.4 

53.9 

Other 

4.2 

5.6 

8.2 

Female students (%) 

48.6 

48.5 

48.5 

Average number of full-time teachers 

49.3 

43.8 

39.9 

Number of schools 

12 

1,452 

9,970 


SOURCE: 2011-2012 data obtained from the U.S. Department of Education 
National Center for Education Statistics Common Core of Data (CCD), Public 
School Universe Data. 


NOTES: The study includes a total of 14 middle schools and 10 high schools. 
Of the 14 middle schools in the study, 2 schools were excluded from analysis 
due to their classification as elementary schools in the 2011-2012 CCD; data 
for grades 6-8 were not available for these two schools. One middle school in 
the study serves grades 6-9. To calculate average enrollment by grade for this 
school, grade 9 enrollment was excluded from the denominator. 

Teacher counts in the CCD are reported in full-time equivalent units, which 
is computed by dividing the amount of time an individual is employed by the 
time normally required for a full-time position. The counts were rounded to the 
nearest whole number. 

a National and study states samples include study schools. Some data were 
missing in the state and national samples due to incomplete CCD data. 
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Table 1.2 


Student Issues and Their Priority for School Leaders and CIS Site Coordinators 


Measure 

Poor 

Academic 

Performance 

Poor 

Attendance 

Behavior/ 

Discipline 

Problems 

High-Risk 

Social 

Behavior 

Lack of Parental 

Involvement/ 

Support 

Family 

Instability 

Students at 

Risk of 

Dropping Out 11 

School leader estimate of the percentage 
of students facing issue* 5 

53.2 

32.3 

28.2 

25.0 

48.2 

26.8 

24.0 

School leaders reporting issue is 
a high priority at school 1 " (%) 

100.0 

86.4 

100.0 

80.0 

85.7 

66.7 

— 

Site coordinator estimate of the 
percentage of students facing issue* 5 

66.2 

36.7 

36.8 

33.2 

60.5 

38.6 

38.2 

Site coordinators reporting issue is a 
high priority for CIS at school 15 (%) 

95.5 

90.5 

100.0 

73.7 

57.1 

50.0 

— 


Number of schools: school leader survey = 22; site coordinator survey = 22 


SOURCES: School leader survey (summer 2014), site coordinator survey (summer 2014). 

NOTES: The percentages presented here include only those respondents who answered the survey item; response rates vary among items. Missing values 
were excluded from calculations. The rates of missing responses to the school leader survey items average to 2 percent and range from 0 percent to 9 
percent, while the rates of missing responses to the site coordinator survey items average to 3 percent and range from 0 percent to 10 percent. 

Respondents at 2 of the 24 study schools did not return a survey. 

“These values are calculated from an individual item on the school leader and site coordinator surveys, asking: “Of all students at your school, 
what percentage would you describe as being at risk of dropping out?” The percentages presented here are calculated from the mean of an 11-point answer 
scale that ranges from 0 = “about 0%” to 10 = “about 100%.” 

b The percentages presented here are calculated from the mean of an 11-point answer scale, where respondents estimated the percentage of students in 
need at their schools from 0 = “about 0%” to 10 = “about 100%.” 

c The survey item was skipped if respondents reported 0 percent of students faced the given issue, or if services relating to the issue were not offered at the 
school. 




Table 1.3 


Student Services As Reported by School Leaders 


Service 

Leaders 

Reporting Service 

Is Offered (%) 

Estimated Percentage 

of Students 
Receiving Service 2 

Leaders Reporting 

Service Is Available 

to All Students (%) 

Average Yearly 

Service 

Frequency 13 

Leaders Reporting 
CIS Is a Key Service 
Provider (%) 

Academic services 

Academic assistance 

90.9 

45.0 

90.0 

36.0 

36.8 

Meeting with an adult staff member to 
discuss academic goals 

86.4 

77.9 

NA 

12.4 

50.0 

Behavior and attendance services 

Behavioral intervention 

81.8 

35.0 

NA 

18.0 

41.2 

Anger management/conflict resolution 

54.5 

25.0 

NA 

16.0 

36.4 

Truancy prevention 

45.5 

24.0 

60.0 

12.3 

60.0 

Social/life-skills services 

Mentoring 

81.8 

17.2 

33.3 

17.0 

82.4 

Gang intervention/prevention 

45.5 

22.0 

60.0 

9.5 

40.0 

Pregnancy prevention 0 

50.0 

36.4 

63.6 

11.6 

72.7 

Bullying prevention 

77.3 

65.3 

82.4 

9.6 

43.8 

Drug abuse prevention 

54.5 

57.5 

75.0 

8.4 

54.6 

Substance abuse support programs 11 

54.5 

4.5 

NA 

5.4 

54.6 

Pregnancy/parent support programs 

54.5 

5.0 

NA 

9.6 

58.3 

Programs for young people involved 
in the juvenile justice system 0 

45.5 

5.6 

NA 

12.3 

22.2 

Grief support programs 

50.0 

6.0 

NA 

7.2 

60.0 

Programs for LGBT young people 

18.2 

10.0 

NA 

10.5 

0.0 
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Table 1.3 (continued) 


Service 

Leaders 

Reporting Service 

Is Offered (%) 

Estimated Percentage 

of Students 
Receiving Service 3 

Leaders Reporting 

Service Is Available 

to All Students (%) 

Average Yearly 

Service 

Frequency b 

Leaders Reporting 
CIS Is a Key Service 
Provider (%) 

Resources to meet basic needs 

Links to providers of basic necessities 

77.3 

28.2 

64.7 

25.7 

76.5 

Physical health screening 

50.0 

42.7 

81.8 

7.1 

30.0 

Exercise class or club/obesity 
intervention and prevention 8 

31.8 

38.6 

85.7 

11.6 

57.1 

College/career preparation 

College planning and preparation 

81.8 

35.0 

50.0 

18.1 

64.7 

Career development/readiness 

72.7 

35.0 

81.3 

19.4 

57.1 

Job shadowing or internship 

40.9 

15.6 

33.3 

4.8 

57.1 

Enrichment/motivation services 

Community service/service learning 

54.5 

16.7 

41.7 

21.0 

54.6 

After- or before-school programs 

68.2 

30.7 

66.7 

30.9 

50.0 

Family-related services 

Family engagement activities 

59.1 

53.8 

100.0 

8.8 

66.7 

Parent group meetings 

54.5 

50.8 

91.7 

4.0 

9.1 

School-sponsored activities for students 

and their families 

63.6 

57.6 

92.9 

6.3 

57.1 

Number of schools 

22 
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Table 1.3 (continued) 


SOURCE: School leader survey (summer 2014). 

NOTES: The values presented in the rightmost four columns include only those surveys in which the 
respondent reported the service was offered at the school and answered the relevant survey item; 
response rates vary among items. Missing values were excluded from calculations. The rates of 
missing responses to the service-availability items (the third column) average to 16 percent, with a 
range of 5 percent to 30 percent. The rates of missing responses to the service-frequency items (the 
fourth column) average to 18 percent, with a range of 5 percent to 50 percent. 

Respondents at 2 out of the 24 study schools did not return a survey. 

LGBT = lesbian, gay, bisexual, and transgender. 

NA = not applicable. 

a The percentages presented here are based on the means of an 11-point scale, where respondents 
estimated the percentages of students in need at their schools from 0 = “about 0%’’ to 10 = “about 
100%.” 

b The values presented here are approximations of the average number of times per year the service 
is offered. Times per year were calculated from respondents’ answers to an original four-point 
frequency scale, where 1 = “once or twice a year” (1.5 times per year), 2 = “less than once a month” 
(4.5 times per year), 3 = “1-2 times a month” (13.5 times per year), and 4 = “one or more times a 
week” (36 times per year). 

c The missing rate for this service-frequency item was 27 percent. 
d The missing rate for this service-frequency item was 29 percent. 
e The missing rate for this service-frequency item was 25 percent. 
r The missing rate for this service-frequency item was 50 percent. 
sThe missing rate for this service-frequency item was 30 percent. 


being generally available in their schools. At least 80 percent of school leaders reported that the 
following services are available: academic assistance, meeting with adult staff members to 
discuss academic goals, behavioral intervention, mentoring, and college planning and prepara¬ 
tion. In schools where these services are offered, leaders reported that half of all students, or 
more, participate in bullying prevention, drug-abuse prevention, meetings with adult staff 
members to discuss academic goals, and school activities and engagement activities with their 
families. Academic assistance was the service offered the most frequently, averaging 36 times 
per year, or about once a week. Other services such as behavioral intervention and mentoring 
were also frequently offered, averaging one or more times a month. School leaders’ responses 
regarding the services available in their schools also suggest that they view CIS as an important 
service provider in their schools, with half of the school leaders or more reporting CIS is an 
important provider of most of these services. 

In addition to school leaders’ survey responses, most school leaders and school support staff 
members stated during interviews that they consider CIS to be an important part of their 
schools’ support systems. Details regarding CIS’s services and work with school staff members 
are included in the following chapter. 
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Structure of This Report 

Chapter 2 presents information about the CIS services available in study schools and provides 
details about how — and with whom — CIS works in these schools. Chapter 3 describes the 
study participants, looks at how much support case-managed students received, and investigates 
whether case-managed and non-case-managed students received different types and amounts of 
support. Chapter 4 presents findings on the effect of CIS case management after two years. It 
first discusses effects on nonacademic outcomes, then turns to effects on attendance, behavior, 
course performance, and school progress. Finally, Chapter 5 highlights conclusions that can be 
drawn from the random assignment component of the evaluation and reflects on the evalua¬ 
tion’s overall findings, taking into account both the random assignment and quasi-experimental 
studies. 
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Chapter 2 


Case Management and Building Communities of Support 

in the Study Schools 


This chapter focuses on Communities In Schools (CIS) service provision and provides details 
about how — and with whom — CIS works in local schools. It begins by reviewing Level 1 
services and the steps of the Level 2 case management process, and then discusses the Level 2 
services available to case-managed students. It then provides details about how CIS works both 
with external partners and school staff to “broker” services for case-managed students in order 
to address those students’ needs. Finally, this chapter seeks to provide a better understanding of 
the support-service contexts in which CIS operates by exploring the work of non-CIS support 
staff members in the schools; how those staff members work with, alongside, and apart from 
site coordinators; and how school support staff members view CIS’s role. 

Highlights from this chapter include: 

• Students in study schools — case-managed students in particular — have 
many support services available to them from a variety of service providers, 
though some services are offered more consistently than others. These ser¬ 
vices include but are not limited to the services provided by CIS site coordi¬ 
nators. 

• CIS’s partners play an important role in providing services to case-managed 
students, though there is variation in how closely partners and site coordina¬ 
tors work together to plan and to provide students with services. While part¬ 
ner service providers allow CIS to meet case-managed students’ needs, it can 
be challenging for CIS to monitor and evaluate the quality of the services 
partners provide. 

• CIS’s model is based on the notion that multiple adult advocates working in¬ 
side and outside schools are needed to provide comprehensive services for 
students with varied academic and social needs. In addition to their outside 
partners, CIS site coordinators work closely with school staff members to 
address the needs of their case-managed students collaboratively. 


Implementing the CIS Model 

CIS program implementation is led at each school by one or more site coordinators responsible 
for all on-site operations. Affiliates determine how many site coordinators to place in a school 
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based on a variety of factors, including the location of a school, its students’ needs, and the 
funding available. Each of the study schools had at least one full-time site coordinator, and half 
had two or more. The site coordinators in these schools were relatively experienced and well 
educated: In the spring of 2014, site coordinators in the study schools had an average of 5.9 
years of experience working in that role and 3.5 years of experience in that role in their current 
schools. In addition, over 65 percent of the site coordinators either had or were pursuing 
graduate degrees, with a majority focused on social work or counseling. 

While this report focuses on Level 2 case management, CIS site coordinators are re¬ 
sponsible for managing all CIS operations in their schools. To understand how site coordinators 
divide their time among Level 1 services, Level 2 case management, and other responsibilities, 
a survey of site coordinators (one of the adult surveys described in Chapter 1) asked respond¬ 
ents to indicate the proportion of their time they spent on each during a typical week. Site 
coordinators reported spending less than 20 percent of their time planning, providing, or 
coordinating Level 1 services. They reported spending approximately 41 percent of their time 
directly planning for or providing Level 2 services and an additional 12 percent of their time 
working with or managing partners who provide Level 2 services. Site coordinators in the study 
reported having an average of 115 Level 2 students on their caseloads during the 2013-2014 
school year. The remaining 27 percent of their time was spent on CIS-related administrative 
work, which includes such tasks as entering data on the services students received into CIS’s 
management information system. 


The Availability of Level 1 Services 

Table 2.1 shows site coordinators’ reports of Level 1 activities, the frequency at which they are 
provided, and the extent to which non-case-managed students receive them. Overall, the 
services available to students cover a broad range of categories, including academics, behavior, 
and attendance. Site coordinators also reported that the majority of services that are available 
are offered at least 10 times a year on average. In addition, approximately a third of the services 
are received by roughly an even split of case-managed and non-case-managed students, and a 
quarter of services are received by more non-case-managed students than case-managed 
students. These proportions indicate that at least some non-case-managed students are making 
use of some CIS support services, even if those services are less intensive than the ones received 
by case-managed students. 
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Table 2.1 


Level 1 Service Provision As Reported by Site Coordinators 


Service 

Site Coordinators Average 

Reporting Service Yearly Service 
Is Offered (%) Frequency” 

Proportion of Students 
Receiving Services Who 
Are Case-Managed 1 ’ 

Academic services 

Academic assistance 

68.2 

30.9 

0.6 

Meeting with an adult staff member to 
discuss academic goals 

59.1 

20.5 

0.6 

Behavior and attendance services 

Behavioral intervention 

54.5 

21.5 

0.5 

Anger management/conflict resolution 

50.0 

17.6 

0.5 

Truancy prevention 1 

40.9 

21.5 

0.3 

Social/life-skills services 

Mentoring 

54.5 

20.9 

0.6 

Gang intervention/prevention d 

40.9 

14.8 

0.6 

Pregnancy prevention^ 

22.7 

7.5 

0.4 

Bullying prevention 

68.2 

15.1 

0.5 

Drug abuse prevention 

59.1 

16.5 

0.4 

Substance abuse support programs 

40.9 

10.5 

0.4 

Pregnancy/parent support programs 

31.8 

17.1 

0.5 

Programs for young people involved in 
the juvenile justice system 

31.8 

8.6 

0.5 

Grief support programs 

27.3 

13.5 

0.5 

Programs for LGBT young people 8 

9.1 

24.8 

0.3 

Resources to meet basic needs 

Links to providers of basic necessities 

77.3 

18.4 

0.7 

Physical health screening 

36.4 

2.6 

0.6 

Exercise class or club/obesity intervention 
and prevention 1 ’ 

18.2 

13.1 

0.4 

College/career preparation 

College planning and preparation 

72.7 

9.3 

0.6 

Career development/readiness 

72.7 

11.0 

0.6 

Job shadowing or internship 

54.5 

6.0 

0.7 

Enrichment/motivation services 

Community service/service learning 

40.9 

10.3 

0.5 

After- or before-school programs 

40.9 

20.5 

0.5 


(continued) 
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Table 2.1 (continued) 


Service 

Site Coordinators Average 

Reporting Service Yearly Service 
Is Offered (%) Frequency 3 

Proportion of Students 
Receiving Services Who 
Are Case-Managed b 

Family-related services 




Family engagement activities 

40.9 

12.5 

0.6 

Parent group meetings 

45.5 

10.1 

0.5 

School-sponsored activities for students 




and their families 

68.2 

6.6 

0.3 

Number of schools 
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SOURCE: Site coordinator survey (summer 2014). 

NOTES: The values presented in the final two columns include only those surveys in which 
respondents reported the service was offered and answered the relevant survey items; response rates 
vary among items. Missing values were excluded from calculations. The rates of missing responses to 
the service-frequency items (the second column) average to 18 percent, with a range of 0 percent to 60 
percent. The rates of missing responses to the case-managed student proportion items (the third 
column) average to 23 percent, with a range of 8 percent to 43 percent. 

Respondents at 2 of the 24 study schools did not return a survey. 

LGBT = lesbian, gay, bisexual, and transgender. 

a The values presented here are approximations of the average number of times per year the service is 
offered. Times per year were calculated from respondents’ answers on a four-point frequency scale, 
where 1 = “once or twice a year” (1.5 times per year), 2 = “less than once a month” (4.5 times per 

year), 3 = “one or two times a month” (13.5 times per year), and 4 = “one or more times a week” (36 

times per year). 

b The values presented here are the means of a three-point scale, where 0 = “mostly non-case- 
managed students,” 0.5 = “an even mix of case-managed and non-case-managed students,” and 1 = 
“mostly case-managed students.” 

c The missing rate for this service-frequency item is 25 percent. 
d The missing rate for this service-frequency item is 25 percent. 
e The missing rate for this service-frequency item is 38 percent. 
f The missing rate for this service-frequency item is 25 percent 
sThe missing rate for this service-frequency item is 60 percent. 
h The missing rate for this service-frequency item is 33 percent. 


Level 2 Case Management 

CIS case management focuses on providing individual attention and guidance to a relatively 
small group of students who are at risk of dropping out. Site coordinators explained that they 
aim to develop trusting relationships with their case-managed students, providing and connect¬ 
ing students with support customized to each one’s needs, creating safe spaces for them to share 
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and discuss their problems, and offering them opportunities to experience new things that they 
might not otherwise (for example, visiting colleges or engaging in community service). There 
are five steps in the case management process: (1) identifying students in need of case manage¬ 
ment, (2) completing assessments to identify students’ needs and set goals, (3) developing case 
plans to address those needs and goals, (4) comiecting students with services or providing them 
directly, and (5) monitoring students’ progress and adjusting services as needed (see Figure 
2.1). As described in more detail in the first report and summarized in Box 2.1, each step of the 
case management process was implemented in the 18 schools that the research team visited 
during the first year of the study, but the details of how each step was implemented varied 
among affiliates. For example, needs assessments were conducted differently at different 
affiliates, site coordinators included varying levels of detail in students’ case plans, and site 
coordinators reviewed students’ school records more or less often for the purpose of formally 
adjusting their case plans. 

As described in Box 2.1, a critical step in the case management process is providing 
services matched to students’ individual needs or connecting students with those services from 
other sources. Table 2.2 summarizes site coordinators’ overall assessments of whether most or 
all of their case-managed students are receiving the services they need to address their individu¬ 
al challenges, and their assessments of the quality of those services. At least two-thirds of site 
coordinators reported that all or most students in need receive services to address poor academic 
performance, poor attendance, and behavior or discipline problems. However, site coordinators 
also acknowledged that for most of the issues students face, the services they receive through 
case management partly address the issue, but more support is needed. 

The Availability of Level 2 Services 

Table 2.3 shows the percentages of site coordinators who reported that a given service 
was offered at their schools during the 2013-2014 school year, as well as the averages of their 
estimates of how often the service was offered. Site coordinators indicated that academic 
services and behavioral support services were among the Level 2 services most frequently 
available to case-managed students in their schools, and that these services were generally 
provided at least once a month. Additionally, about two-thirds of site coordinators reported 
providing truancy-prevention activities at least twice a month to students struggling with 
attendance. Some coordinators also reported offering case-managed students other services and 
fomis of support with relative consistency and frequency, including mentoring, college planning 
and preparation, community service and service-learning opportunities, and family-engagement 
activities, as well as resources to meet their basic needs. 
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Figure 2.1 

The CIS Case Management Process 




CIS also supports case-managed students who might benefit from social and life-skills 
services. Some of these social and life-skills services were provided more frequently than 
others. For example, more than half of site coordinators indicated that they provided mentoring, 
gang prevention, and bullying prevention. However, fewer than half reported providing other 
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Box 2.1 


The CIS Case Management Process in More Detail 

Student identification. The CIS case management process begins with the identification of at- 
risk students who would benefit from additional services. Site coordinators gave relatively 
consistent descriptions of how these students are identified: Site coordinators encourage 
teachers, administrators, and other student support staff members on campus to make student 
referrals and recommendations. Some CIS affiliates also have their own guidelines for identi¬ 
fying eligible students, or follow district-wide guidelines. For example, one affiliate has 
contractual agreements with local agencies such as the city housing authority and the foster 
care system that require site coordinators to enroll at least a certain number of students served 
by those agencies. Students also sometimes simply choose to enroll in the program. 

Needs assessment. After identifying students to receive case management, site coordinators 
conduct individual assessments to leam more about those students’ needs. Site coordinators at 
almost all schools explained that they review school data such as attendance records, grades, 
and behavioral incident records to better understand their students’ needs. They also reported 
using a variety of standardized assessment tools to measure students’ levels and areas of need. 
During the two years of the study, site coordinators indicated in interviews and surveys that the 
issues their students most commonly faced were poor academic performance, lack of parental 
involvement, poor attendance, behavior problems, family instability, and high-risk social 
behavior (for example, drug use or gang participation). 

Case planning. After conducting the individual needs assessment, site coordinators create 
individual case plans that document each student’s areas of need, goals, and service plans for 
the school year. Most site coordinators reported that students actively participate in the devel¬ 
opment of their case plans. 

While all of the schools in the study appear to document this case plan in some way, site 
coordinator and affiliate interviews indicate that there is variation in the levels of effort and 
detail involved. In one affiliate, for example, site coordinators suggested that a case plan 
developed early in the year does not fully reflect the depth of a student’s needs and is more of a 
formality than a useful document to guide student services, while in two other affiliates staff 
members and site coordinators described much more specific, detailed procedures for develop¬ 
ing case plans. 

Service provision. Based on their students’ needs assessments and subsequent case plans, site 
coordinators provide Level 2 services and connect students with Level 2 services from other 
providers. Site coordinators reported spending the greatest amount of their time on planning 
for or providing Level 2 services, and they said during interviews that their responsibilities to 
case-managed students are a priority over their Level 1 duties. The services provided to case- 
managed students during the second year of the study are described in the main text. 

Monitoring student progress. The final step of the case management process involves moni¬ 
toring student progress and adjusting the services that students receive. These last two steps in 
the process are cyclical, and may take place numerous times throughout the school year. In all 

(continued) 
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Box 2.1 (continued) 

schools site coordinators indicated that they have access to student data and that they use it 
to track students’ progress. Site coordinators in almost all schools visited said they moni¬ 
tored their case-managed students at least quarterly, primarily by checking grades and 
attendance and writing progress reports about them. Most site coordinators said they had 
access to student data in “real time,” while some said they only had access to students’ 
records at the end of each marking period. Site coordinators also reported that they rely on 
teachers, counselors, and other school staff members to help them monitor student progress. 


types of social and life-skills services such as programs for young people involved in the justice 
system. Those who did report offering these services suggested that they provide them less than 
twice a month. 

Table 2.4 provides more specific infonnation about who provides these same services 
to case-managed students. As described previously, CIS provides some services itself but also 
draws upon existing community and school resources. Site coordinators reported that many of 
the services included in Table 2.3 are directly provided either by themselves or other CIS staff 
members. But Table 2.4 also indicates that CIS partners and school staff members play a 
substantial role in providing Level 2 services. 

CIS Partnerships 

Because CIS partners and school staff members play such an important role in provid¬ 
ing Level 2 services to case-managed students, the research team sought to leam more about 
CIS’s partnerships during the second year of the study by including additional questions about 
partnerships in interviews with CIS staff members and by conducting interviews with CIS 
partners. Site coordinators in all of the affiliates in the study described the “brokering” of 
student support services as an essential part of their work: When site coordinators leam about 
their students’ individual needs, they connect those students with partners from the community 
who can provide resources or services to address those needs. When a CIS site coordinator 
plays the role of “broker” in this way, these services are “counted” as Level 1 or Level 2 
services, depending on the nature of the service. For many of the types of services listed in 
Table 2.4, site coordinators explained in interviews that the support they provide directly may 
be different from that provided by partners or school staff members. For example, many site 
coordinators said that they may connect students with teachers or tutors who provide tutoring, 
and if they directly provide services to address students’ academic needs, they may be less 
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Table 2.2 


Case-Managed Student Issues and Service Details As Reported by Site Coordinators 


Measure 

Poor 

Academic 

Performance 

Poor 

Attendance 

Behavior/ 

Discipline 

Problems 

High-Risk 

Social 

Behavior 

Lack of Parental 

Involvement/ 

Support 

Family 

Instability 

Respondents’ estimate of the percentage of 
case-managed students facing this issue a 

60.9 

35.9 

35.9 

28.6 

54.5 

33.8 

Site coordinators reporting most or 
all students in need are served (%) 

72.7 

66.7 

75.0 

42.1 

47.6 

60.0 

Service-quality rating b 

0.6 

0.6 

0.7 

0.6 

0.5 

0.5 

Number of schools 
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SOURCE: Site coordinator survey (summer 2014). 

NOTES: The values presented include only those surveys in which the respondent answered the survey item; response rates vary among 
items. Items for the second and third measures were skipped if respondents reported “about 0%’’ of case-managed students faced the given 
issue at their school, or if services relating to the issue were not offered at the school. Missing values were excluded from calculations. 

The rates of missing responses to the above measures average to 2 percent, with a range of 0 percent to 18 percent. 

Respondents at 2 of the 24 study schools did not return a survey. 

a The percentages presented here are calculated from the mean of an 11-point answer scale, where respondents estimated the percentage 
of students in need at their school from 0 = “about 0%” to 10 = “about 100%.” 

b The values presented here are means on a three-point quality scale, where 0 = “services don’t address this issue, services are not 
appropriate for need,” 0.5 = “services partially address this issue, more support is needed,” and 1 = “services address this issue well.” 



Table 2.3 


Case-Managed Service Provision and Frequency 
As Reported by Site Coordinators 


Service 

Service Is Offered 

at School (%) 

Average Yearly 
Service Frequency 6 

Academic services 

Academic assistance 13 

90.9 

28.0 

Meeting with an adult staff member to 
discuss academic goals 

90.9 

15.0 

Behavior and attendance services 

Behavioral intervention 1 ’ 

90.9 

25.9 

Anger management/conflict resolution 

86.4 

23.1 

Truancy prevention 

63.6 

23.3 

Social/life-skills services 

Mentoring 

90.9 

24.2 

Gang intervention/prevention 

59.1 

20.6 

Pregnancy prevention 

59.1 

16.9 

Bullying prevention 

72.7 

17.1 

Drug abuse prevention 1 ’ 

72.7 

23.9 

Substance abuse support programs 

40.9 

18.8 

Pregnancy/parent support programs 

40.9 

12.8 

Programs for young people involved in 
the juvenile justice system 13 

36.4 

18.9 

Grief support programs’ 1 

54.5 

17.5 

Programs for LGBT young people 6 

13.6 

13.5 

Resources to meet basic needs 

Links to providers of basic necessities 

95.5 

18.9 

Physical health screening 

54.5 

6.8 

Exercise class or club/obesity intervention 
and prevention 

36.4 

17.6 

College/career preparation 

College planning and preparation 

95.5 

12.1 

Career development/readiness programs 

86.4 

12.5 

Job shadowing or internship 

68.2 

10.0 

Enrichment/motivation services 

f 

Community service/service learning 

77.3 

10.7 

After- or before-school programs 13 

68.2 

27.9 


(continued) 
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Table 2.3 (continued) 


Service 

Service Is Offered 

at School (%) 

Average Yearly 
Service Frequency 3 

Family-related services 



Family engagement activities 

90.9 

8.3 

Parent group meetings 

68.2 

7.2 

School-sponsored activities for students 



and their families 

72.7 

9.5 

Number of schools 
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SOURCE: Site coordinator survey (summer 2014). 

NOTES: Response rates vary among items. Missing values were excluded from 
calculations. The rates of missing responses to the service-frequency items average to 18 
percent, with a range of 0 percent to 50 percent. 

Respondents at 2 of the 24 study schools did not return a survey. 

LGBT = lesbian, gay, bisexual, and transgender. 

a The values presented here are approximations of the average number of times per 
year the service is offered. Times per year were calculated from respondents’ answers on 
a four-point frequency scale, where 1 = “once or twice a year” (1.5 times per year), 2 = 
“less than once a month” (4.5 times per year), 3 = “one or two times a month” (13.5 
times per year), and 4 = “once or more per week” (36 times per year). 

b Of the site coordinators who reported that this service is offered, more than 50 
percent reported that this service is offered once or more per week. 
c The missing rate for this service-frequency item is 30 percent. 
d The missing rate for this service-frequency item is 25 percent. 
e The missing rate for this service-frequency item is 50 percent. 

'The missing rate for this service-frequency item is 28 percent. 


likely to tutor students and more likely to do things like checking and discussing grades with 
students or talking to teachers about students’ academic progress. 

For 11 of the 26 services covered in the survey, at least 50 percent of site coordinators 
indicated that partners play a role in providing the service in their schools. In particular, partners 
were highly involved in mentoring, college and career preparation activities, and drug abuse 
prevention activities. For example, nearly 75 percent of site coordinators reported that partners 
provided mentoring services. In their interviews, site coordinators explained that they use 
mentors from local colleges, universities, and community-based nonprofit organizations, along 
with individuals who apply directly to the affiliate. Drawing on partners to help provide services 
also allows CIS to offer a greater variety of services to students, as community partners often 
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Table 2.4 


Providers of Services for Case-Managed Students 
As Reported by Site Coordinators 


Service 

Service Is Provided 

by CIS Staff (%) 

Service Is Provided 

by CIS Partners (%) 

Service Is Provided 

by School Staff (%) 

Academic services 

Academic assistance 

72.7 

54.6 

54.6 

Meeting with an adult staff member to 
discuss academic goals 

81.8 

36.4 

36.4 

Behavior and attendance services 

Behavioral intervention 

81.8 

54.6 

40.9 

Anger management/conflict resolution 

77.3 

50.0 

36.4 

Truancy prevention 

59.1 

9.1 

31.8 

Social/life-skills services 

Mentoring 

68.2 

72.7 

18.2 

Gang intervention/prevention 

40.9 

27.3 

13.6 

Pregnancy prevention 

45.5 

31.8 

22.7 

Bullying prevention 

45.5 

45.5 

31.8 

Drug abuse prevention 

54.6 

59.1 

18.2 

Substance abuse support programs 

22.7 

36.4 

13.6 

Pregnancy/parent support programs 

36.4 

36.4 

9.1 

Programs for young people involved in 
the juvenile justice system 3 

22.7 

31.8 

4.6 

Grief support programs' 3 

18.2 

18.2 

13.6 

Programs for LGBT young people 3 

4.6 

4.6 

0.0 

Resources to meet basic needs 

Links to providers of basic necessities 

86.4 

50.0 

27.3 

Physical health screening 

22.7 

27.3 

27.3 

Exercise class or club/obesity intervention 
and prevention 

18.2 

13.6 

4.6 

College/career preparation 

College planning and preparation 

86.4 

59.1 

45.5 

Career development/readiness programs 

77.3 

59.1 

36.4 

Job shadowing or internship 

50.0 

50.0 

13.6 

Enrichment/motivation services 

Community service/service learning 11 

68.2 

31.8 

13.6 

After- or before-school programs 

54.6 

50.0 

22.7 


(continued) 
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Table 2.4 (continued) 


Service 

Service Is Provided 

by CIS Staff (%) 

Service Is Provided 

by CIS Partners (%) 

Service Is Provided 

by School Staff (%) 

Family-related services 




Family engagement activities 

77.3 

45.5 

50.0 

Parent group meetings 

54.6 

40.9 

40.9 

School-sponsored activities for students 




and their families 

59.1 

50.0 

50.0 

Number of schools 
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SOURCE: Site coordinator survey (summer 2014). 

NOTES: Response rates vary among services. Missing values were included in calculations. The rates of 
missing responses average to 18 percent, with a range of 0 percent to 50 percent. 

Respondents at 2 of the 24 study schools did not return a survey. 

LGBT = lesbian, gay, bisexual, and transgender. 
a The missing rate for this item is 30 percent. 
b The missing rate for this item is 25 percent. 
c The missing rate for this item is 50 percent. 
d The missing rate for this item is 28 percent. 


work with CIS on drug abuse prevention, bullying prevention, after-school programs, and job 
shadowing. 

Connecting students with partner adults who can help address their needs means that 
students are ultimately making more connections with more adults, and that more adults are 
aware of and actively working with them. Just as site coordinators build and maintain relation¬ 
ships with students, they also build and maintain relationships with partners, and help partners 
nurture their own relationships with case-managed students. Moreover, partners may allow site 
coordinators to focus attention and resources where they feel attention and resources are really 
needed. As one site coordinator explained: 

I think there’s wonderful opportunity for symbiosis [with partnerships]. I see 
these groups that are in the community wanting to reach out to populations 
and this is an opportunity that I have to present them with the population. 

And it’s also beneficial for me not to have to create and provide all of the 
programming.... I think, “Why not tap into those resources, other ideas, other 
people bringing different energy?” 
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Partnership Formation and Monitoring 

Site coordinators in all affiliates described partnering extensively with other nonprofit 
organizations that concentrate on children and young people to provide Level 2 services, such 
as Girl Scouts of America, YMCA/YWCA, and AmeriCorps. CIS staff members and staff 
members from partner organizations reported that the affiliate office often reaches out to 
establish partnerships initially, while site coordinators maintain the day-to-day relationship with 
partner staff members in schools. Some described instances in which site coordinators estab¬ 
lished partnerships themselves through personal connections, and in some cases, site coordina¬ 
tors identify organizations with which they are interested in partnering, and affiliate-level staff 
members make the initial connections to those organizations. CIS staff members also explained 
that representatives from outside organizations reach out to CIS themselves to establish partner¬ 
ships. 


The degree to which partners are formally and operationally integrated into CIS’s work 
appears to vary from partner to partner, from school to school, and from affiliate to affiliate. CIS 
staff members discussed having formal agreements with at least some of their partner organiza¬ 
tions, but not with all partners. In addition, some partners are more likely than others to be 
granted autonomy in planning and delivering services, and some affiliates are more likely than 
others to grant it. A site coordinator may refer a student to a partner service provider when the 
site coordinator feels that the student’s particular needs are best handled by an external profes¬ 
sional (for example, if the student is pregnant or suicidal). In those cases the site coordinator 
may not work closely with the partner to plan the specifics of the service once the connection is 
made. In other cases — especially if a partner organization is relatively well integrated into the 
school — the site coordinator may collaborate with the partner to plan the details of service 
provision. For example, one partner staff member said that she meets monthly with her school’s 
site coordinator to discuss service provision and to brainstonn solutions to any issues that may 
have arisen with the case-managed students receiving her services. Similarly, another site 
coordinator indicated that he meets with tutors regularly to discuss students’ progress and adjust 
their tutoring plans as needed. Given the different backgrounds and needs of case-managed 
students, it is perhaps not surprising that the levels of involvement and the roles of these 
external partners vary. 

In addition to describing this variation in the integration of external partners, in inter¬ 
views CIS staff members also suggested that it is challenging to evaluate the quality of the 
partners’ services themselves. Site coordinators have many responsibilities, and may not have 
time to assess whether all the services being provided to case-managed students are of high 
quality. Site coordinators reported that there is not a formal way of evaluating how well the 
partner-provided services are working. Most site coordinators indicated that they do not receive 
training or guidance about how to assess the effectiveness or quality of partner-provided 
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services. They said that they rely on students telling them whether or not the services are going 
well, and that they generally make conclusions about the effectiveness of those services based 
on students’ outcomes. 


School Staff Members and CIS 

As discussed in Chapter 1, CIS operates in high-poverty schools whose students face challenges 
that may threaten their academic success (see Tables 1.1 through 1.3). While school leaders 
indicated that CIS plays an important role in providing support to students, there are often many 
other services available in the school. During the second year of the study, the research team 
sought to gain a better understanding of the school contexts in which CIS operates by learning 
more about the work of non-CIS support staff members, how CIS works with school staff 
members (including with regard to case-managed students), and how school support staff 
members view CIS. 

School Support Staff Members’ Roles 

Nationally, schools rely on a mix of guidance counselors, social workers, and similar 
staff members to address students’ needs. Nearly all school leaders (95 percent) reported having 
guidance counselors in their schools, 68 percent reported having social workers, and 18 percent 
reported having other support staff members (cited examples include graduation coaches and 
leaming-lab support specialists). In schools that had these support positions, school leaders 
indicated having an average of 3.1 full-time equivalent (FTE) guidance counselors, 1.1 FTE 
social workers, and 2 FTE other support staff members. 1 Across all the schools in the sample, 
the average number of support staff members was 4.3 FTEs, not including the CIS site coordi¬ 
nator. 


School leaders reported that during the 2013-2014 school year, guidance counselors, 
social workers, and other support staff members had caseloads averaging 323 students, 420 
students, and 400 students per FTE, respectively. During interviews, many school leaders said 
that these caseload sizes are one reason they turn to CIS to supplement the services available to 
their students. They also suggested that these large caseloads may prevent staff members from 
devoting the time and attention it would take to fully address all of an individual student’s 
needs. In addition, guidance counselors noted that the challenge is exacerbated by the wide 
variety of services they are responsible for providing: monitoring academic progress and 


'FTEs indicate the number of full-time employees at an organization plus the number of part-time em¬ 
ployees, standardized to a full-time basis. For example, an organization with 4 full-time and 3 half-time 
employees would have 5.5 FTEs. 
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behavioral needs, handling testing, addressing crises, tracking progress to graduation, and 
leading the school’s student support team. 

School Staff Members and Level 2 Services 

In all schools in the study, interviewees reported that CIS staff members regularly work 
with the school’s staff, especially teachers and support staff members. Just as site coordinators 
rely on CIS’s partners to provide some services to case-managed students, they also work with 
the other adults in their schools. School staff members frequently provide services to address the 
types of challenges faced by case-managed students, and site coordinators often draw on those 
services. As shown in the final column of Table 2.4, over half of the site coordinators surveyed 
indicated that school staff members provide academic assistance to case-managed students, and 
more than one-third said that school staff members meet with case-managed students to discuss 
setting academic goals. 

Many site coordinators also reported that other school staff members provided behav¬ 
ioral, attendance, college and career, and family-related services to their case-managed students, 
as detailed in Table 2.4. Over one-third of site coordinators responded that school staff members 
provided behavioral interventions and anger-management services, and nearly one-third noted 
that they conducted truancy prevention. Between one-third and half of site coordinators also 
reported that school staff members provide college preparation and career development support 
to case-managed students. Finally, half of all surveyed CIS site coordinators indicated that 
school staff members coordinate family-engagement activities attended by case-managed 
students, or organize other school-sponsored activities for students and their families. 

As indicated above, when CIS staff members connect case-managed students to ser¬ 
vices provided by school staff members (and partners), these services “count” as Level 2 
services — even if the services are also available to non-case-managed students. In other words, 
one part of a site coordinator’s job is to make sure case-managed students take advantage of the 
services already available to them. 

Interview data provide more evidence that CIS staff members regularly interact with 
school staff members about their case-managed students, checking on their academic and 
behavioral progress. Interviewees from all schools described discussions with teachers and 
counselors to assess how well case-managed students were doing in classes, and in several 
schools it appeared that the site coordinators serve as the conduits through which information 
passes between teachers and students. For example, some teachers provided site coordinators 
with additional worksheets to complete with case-managed students who were falling behind on 
assignments or who needed tutoring. In other schools, site coordinators advocate for students 
who are struggling academically and who may have difficulty articulating what assistance they 
need from their teachers. One site coordinator noted that as a result of such discussions, teachers 
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allow one of her case-managed students to do class work in the CIS office during class time, 
provided that the student turns in that work at the end of class. This accommodation allows the 
student to receive personal attention in a quieter space than the classroom would offer. Site 
coordinators from one affiliate also shared that teachers have asked them to attend class periodi¬ 
cally to help with a lesson or with classroom management, or to observe a case-managed 
student’s behavior as a first step toward deciding how to improve it. 

Interview data also suggest that staff members at most schools help CIS site coordina¬ 
tors forge stronger connections with students’ parents and family members. At some schools, 
for example, guidance counselors and teachers sometimes invite site coordinators to participate 
in parent conferences so they can comment together about their attendance checks and behav¬ 
ioral monitoring of case-managed students. In other schools, the site coordinators and school 
staff members coordinate parent advocacy and involvement groups in order to set up home 
visits together or in-school meetings where parents can hear from teachers, site coordinators, 
and guidance counselors. 

Additional Work with School Staff Members 

Interviewees at almost all schools indicated that site coordinators and school support 
staff members cooperate to put on school social events such as awards assemblies, graduation 
parties, movie nights, and literacy nights. Site coordinators and guidance counselors at several 
schools also said that together they coordinate programs that provide basic resources for both 
case-managed and non-case-managed students, such as food pantries or on-campus closets with 
school supplies and uniforms. 

Finally, most of the study schools organize intervention teams composed of guidance 
counselors, social workers, administrators, and teachers — and often CIS site coordinators — 
who meet regularly to align or divide up services to students based on their needs. Site coordi¬ 
nators from all schools the research team visited in three affiliates described being part of these 
teams, which often meet at least once a month. During these meetings, members of the interven¬ 
tion teams confer about individual students’ needs and, depending on the school, may either 
plan to collaborate to have a student receive services from multiple adults (for example, the site 
coordinator and the social worker) or decide which students will be served by whom. In 
addition, interviewees reported that several intervention teams also meet to organize school¬ 
wide activities or events, such as field trips and parental outreach sessions. 

School Staff Members’ Perceptions of CIS 

When asked about what they consider to be the primary roles of CIS at their schools, 
guidance counselors, social workers, and school administrators generally said that site coordina¬ 
tors provide additional support to students, especially those who need the most help academical- 
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ly and behaviorally. At the same time, site coordinators described some confusion among 
school staff members about CIS’s role, and said that the process for referring students to CIS 
needed to be streamlined. One site coordinator, for example, reported creating an instructional 
flow chart that outlined different referral protocols and encouraged the school’s administration 
team and teaching staff to use it. 

In all schools and affiliates, school staff members cited site coordinators’ ability to form 
personal relationships with students as an important benefit of having CIS in the school. School 
administrators most often said that CIS’s primary role is to establish emotionally supportive 
relationships with students, and often said CIS site coordinators have responsibilities similar to 
those of their guidance counselors, social workers, and behavior specialists. Interviewees also 
pointed out, though, that CIS site coordinators had more time and flexibility to reach students 
personally in ways other support staff members could not. In fact, both CIS site coordinators 
and school support staff members said that this additional time allows site coordinators to forge 
closer and more sustainable relationships with students. Guidance counselors at several schools 
said that they had to focus on grades, graduation credit checks, or class scheduling, and to do so 
for relatively larger caseloads of students than site coordinators. In most cases, guidance 
counselors’ impressions were that CIS site coordinators had the time to engage students in more 
personal interactions that were not solely focused on administrative tasks or scheduling because 
they did not have these constraints. 


Discussion 

Much of this chapter focused on the provision of case-managed services by CIS staff members, 
partners, and other school support staff members. While site coordinators provide many 
services directly, they also connect students to a range of other service providers within their 
schools and around their communities. They aim to serve as advocates for their case-managed 
students, which often involves making sure they take advantage of the services already offered 
in their schools. Site coordinators regularly described partnerships as an essential way to 
maximize the number of students they can assist and the types of services they can offer them. 
While site coordinators see value in having a range of services available for students, it may 
also be valuable to improve the processes through which site coordinators can monitor the 
quality of those services, and to help them connect students to those services more efficiently. 
The findings presented in this chapter suggest that CIS staff members receive varying amounts 
of guidance regarding how to monitor the services provided by partners, and that there is not a 
uniform method for evaluating those services’ effects on students. CIS may be able to promote 
high-quality services in part by exploring ways to monitor, evaluate, and create new strong 
partnerships. 
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CIS works closely with other school support staff members such as guidance counselors 
and behavior specialists, and those support staff members value CIS as an important contributor 
to the pool of adults who address students’ needs. However, drawing distinctions between the 
roles and responsibilities of site coordinators and those of other support staff members may in 
some instances be a challenge. CIS could address misconceptions about site coordinators’ roles 
by providing additional information and guidance to schools about those roles and responsibili¬ 
ties, and about the various ways students may come to receive support services. 
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Chapter 3 

Study Students and Support Services Received 


This chapter focuses on the students who participated in the study and the support services they 
received during the 2013-2014 school year. Using data from school records, student surveys, 
and Communities In Schools (CIS) management information systems, this chapter begins with 
a description of the case-managed and non-case-managed students in the study. While the 
previous chapter describes the services available to students in the study schools, this chapter 
provides details about the services received by both case-managed and non-case-managed 
students during the year. This information is useful for understanding the difference between 
case management and “business as usual” for similar students in the study schools. In addition, 
this chapter includes an investigation of whether the highest-risk students (that is, those with a 
recent history of course failures, suspensions, and very poor attendance) received different or 
more services than moderate-risk students. 

Highlights from this chapter include: 

• As was the case in the first year of the study, the students in the case- 
managed and non-case-managed groups for the second year of the study had 
similar baseline characteristics. 

• During the second year of the study, approximately 80 percent of the students 
in the case-managed group received services. Those students received ser¬ 
vices an average of nearly 20 times during the year, which amounted to an 
average of just over 18 total Level 2 service hours. Nineteen percent of the 
students in the case-managed group did not receive any services. 

• Compared with non-case-managed students, case-managed students reported 
participating in support activities more frequently, suggesting that CIS case 
management is providing something above and beyond what students other¬ 
wise receive. Specifically, case-managed students were significantly more 
likely to report participating in individual and group meetings with adults in 
school, meeting with mentors, and receiving tutoring. Nonetheless, even 
those students not receiving CIS Level 2 case management were getting 
some support services, as opposed to being a “no-service” comparison group. 

• There were significant differences in the services received by case-managed 
students based on their classification as being at either high or moderate risk 
of dropping out. Specifically, a smaller proportion of high-risk case-managed 
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students received services during the second year of the study than did mod¬ 
erate-risk students: 75 percent of high-risk students received services com¬ 
pared with 85 percent of moderate-risk students. Further, in most service cat¬ 
egories a greater proportion of moderate-risk students received services than 
did high-risk students. 


Student Samples 

The schools included in this study serve students facing many challenges that may hinder their 
progress in school. Before and during this study, CIS and school staff members emphasized that 
a large proportion of the students in their schools were in need of the kind of supplementary 
support provided to Level 2 case-managed students. As described in Chapter 1, students were 
assigned at random to either receive or not receive case management because CIS did not have 
the resources to serve all the students in need in the study schools. 

To assess the effect of CIS case management, the analyses in this report discuss two 
samples of students. The stable school records sample, which includes a total of 1,501 students, 
is used to examine students’ demographic characteristics and outcomes that can be measured 
using their school records (for example, grades and attendance). Students are included in this 
sample if they had school records data for both Year 1 and Year 2 of the study. 1 The second 
sample is the stable student survey sample, which includes a total of 1,416 students who 
responded to both the spring 2013 (Year 1) and spring 2014 (Year 2) surveys. 2 This sample is 
used to examine students’ responses to survey questions related to their backgrounds and 
experiences in and out of school. 3 

Table 3.1 examines students’ baseline demographic characteristics and shows that the 
students in the case-managed and non-case-managed groups included in the stable school 
records sample were similar when they enrolled in the study. (See Box 3.1 for an explanation of 
how to interpret the baseline and effect tables in this report.) Overall, the students were 


Yhe stable school records sample represents 85 percent of all students in the study sample who were eli¬ 
gible to participate in the second year of the study (that is, the 1,764 students who had not graduated from high 
school and were enrolled in the 24 schools that agreed to participate in the study for a second year). See 
Appendix B for further information on response rates. 

2 The stable student survey sample represents 80 percent of all students in the study sample who were eli¬ 
gible to participate in the second year of the study (see previous note). See Appendix B for further information 
on response rates. 

3 The two samples mostly overlap: 89 percent of students in the stable school records sample are in the 
stable student survey sample, and 95 percent of students in the stable student survey sample are in the stable 
school records sample. 


42 



Box 3.1 


Reading and Interpreting the Findings in the Report Tables 

Many of the tables in this report show the baseline characteristics, services received, or student 
outcomes of students in the case-managed and non-case-managed group — as well as the 
difference between these two groups of students. The values presented in these tables are 
presented as follows: 

“Case-managed” column: This column shows data for the students randomly assigned to 
receive case management. This column reflects the outcomes of the average student in the 
analysis sample. 

“Non-case-managed” column: This column shows data for the students randomly assigned 
to the non-case-managed group. These values are intended to represent what the outcomes of 
the students in the case-managed group would have been had they not been assigned to case 
management. 

“Estimated effect” or “estimated difference” column: This column shows the difference 
between students in the case-managed and non-case managed group with respect to their 
baseline characteristics, the services that they received, or their outcomes. The values in this 
column should be interpreted as the estimated effect or impact for the average student in the 
analysis sample. Statistical significance is a measure of the degree of certainty one may have 
that a program’s effect is actually nonzero. If an effect estimate is statistically significant, then 
one may conclude with some confidence that the program really had an effect on the outcome 
being assessed. If an effect estimate is not statistically significant, then the nonzero estimate is 
more likely to be a product of chance. In this report, statistical significance is based on a 
significance level of 10 percent, and in these tables, statistical significance of the estimated 
effect or impact is indicated (*) when the p-value is less than or equal to 10 percent. 

“Effect size” (Chapter 4 and Appendixes A and C): This column shows the estimated effect 
scaled as an effect size. The “effect size” is a metric that is widely used for gauging whether 
the magnitude of a program’s effect is large or small. It is defined as the estimated effect of a 
program (or the difference in outcomes between case-managed and non-case-managed stu¬ 
dents) divided by the standard deviation of the outcome of interest. For example, an effect size 
of 0.20 represents an improvement in student outcomes that is equal to 20 percent of the 
standard deviation of the student-level distribution for that particular outcome. The effect size, 
therefore, provides an indication of how much CIS improved a student’s outcomes relative to 
where they would have been in the outcome distribution for students in the program’s target 
population. In this report, effect sizes are calculated based on the standard deviation of the 
outcome of interest for students in the non-case-managed group. The standard deviation for the 
non-case-managed group reflects the expected variability in the outcome that one would find 
in the absence of CIS. 
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Table 3.1 


Students’ Baseline Characteristics 


Baseline Characteristic (%) 

Case- 

Managed 

Group 

Non-Case- 

Managed 

Group 

Estimated 

Difference 

P-Value for 

Estimated 

Difference 

Race/ethnicity 



* 

0.087 

Hispanic 

60.0 

60.7 

-0.7 


Black, non-Hispanic 

35.1 

31.8 

3.4 


White, non-Hispanic 

2.6 

4.0 

-1.4 


Asian 

1.5 

2.2 

-0.7 


Other 

0.6 

1.4 

-0.7 


Male 

43.7 

42.9 

0.8 

0.753 

Eligible for free or reduced-price lunch' 1 

47.2 

44.7 

2.5 

0.119 

English as a second language 

13.9 

13.1 

0.8 

0.640 

Qualified for a gifted program 

2.4 

3.2 

-0.8 

0.375 

Chronically absent 

8.7 

8.8 

0.0 

0.981 

Average attendance rate 

95.7 

95.6 

0.1 

0.714 

Failed at least 1 core course 

21.8 

21.9 

-0.1 

0.973 

Average core course marks 

80.8 

80.9 

0.0 

0.909 


Joint test of difference between groups b 

(X 2 =18.6) 


0.667 

Number of students (total = 1,501) 

751 

750 



SOURCE: MDRC calculations based on student records obtained from school districts. 

NOTES: The analyses reported in this table are based on the stable school records sample, which 
includes all students with course-failure data for the 2012-2013 and 2013-2014 school years. Due to 
small numbers, percentages for the Native American demographic group are not included in the 
table. 

The estimated differences between the case-managed group and the non-case-managed group are 
regression-adjusted using ordinary least squares, controlling for random assignment blocks by 
school. The values in the column labeled “Case-Managed Group” are the observed means for 
students randomly assigned to the case-managed group. The “Non-Case-Managed Group” values 
are the regression-adjusted means for students randomly assigned to the non-case-managed group, 
using the observed distribution of the case-managed group across random assignment blocks as the 
basis for the adjustment. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between the case-managed and non-case-managed 
groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 
percent. 

(continued) 
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Table 3.1 (continued) 


Due to missing values, the number of students included varies by characteristic. The average 
percentage of missing data for any given characteristic is 14 percent and ranges from 0 percent 
(gender) to 37.7 percent (free or reduced-price lunch eligibility). The percentage of missing data is 
high for free or reduced-price lunch eligibility because data for this characteristic were not 
available for students in two of the six districts in the stable school records sample. 
a Data for this characteristic were missing for 37.7 percent of the sample. 
b A chi-square test was used to determine whether there is a systematic difference between the 
case-managed group and the non-case-managed group when they joined the study, based on the 
characteristics included in this table as well as indicators of missing data for all relevant student 
characteristics. 


predominantly minority and from relatively low-income families — more than 90 percent were 
black or Hispanic and nearly 50 percent were eligible for free or reduced-price lunches. Ap¬ 
proximately 60 percent were female. The baseline student surveys suggest that students in the 
case-managed and non-case-managed groups also had similar family characteristics, education¬ 
al aspirations, engagement with school, and adult support at home, at school, and outside of 
home and school, (see Table 3.2). 

Overall, the supplemental infonnation students provided on baseline surveys confimis 
that many of them face challenges in their lives. For example, fewer than 60 percent of students 
reported living with both parents, approximately one-third of the students with older siblings 
reported that a sibling had dropped out of high school, and at least half of students either 
reported that their mother or father (or both) did not complete high school or that they did not 
know their parents’ educational attainment. 


Services Received by Case-Managed Students 

This section provides details about the types, frequencies, and amounts of services received by 
CIS case-managed students (see the “Activities” column in Figure 1.1, the CIS case manage¬ 
ment logic model). The research team learned these details from management infonnation 
system data documented by CIS site coordinators. 4 Whenever a case-managed student receives 


4 The research team received management information system data for students in the case-managed 
group. The CIS national office provided information for two affiliates using the national database, and two 
affiliates provided data from their own or state systems. To compare service-provision information from these 
three different data systems, the research team collapsed the data into the eight service types described below. 
However, not all affiliates had information regarding all eight categories. These are the best estimates of 
service receipt that can be derived from these data, but due to the inconsistencies in the ways the three systems 
gather data, those estimates may over- or understate how much service students in the sample received. 
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Table 3.2 


Selected Student Responses on the Baseline Survey 

Case- Non-Case- P-Value for 

Managed Managed Estimated Estimated 
Survey Item Group Group Difference Difference 


Household makeup 3 (%) 

Lives with 2 or more parents/guardians 
Lives with 1 parent/guardian 
Lives with 1 or more grandparent(s) 

Lives with his/her own child 

Language predominantly spoken at home b (%) 

English 
Not English 

Parent educational attainment (%) 

Father 

Not a high school graduate 
High school graduate or equivalent 
College graduate or higher 
Don’t know 
Mother 

Not a high school graduate 
High school graduate or equivalent 
College graduate or higher 
Don’t know 

Did any siblings leave high school before graduation?' 
None left high school 
At least 1 left high school 

Student engagement with school (l-4) d 

How far would you like to go in school? (%) 

Some high school 
Finish high school 

Some college or trade/technical school 
Finish college or trade/technical school 
Graduate school after college 
Don’t know 


58.8 

56.6 

2.3 

0.398 

34.3 

36.5 

-2.2 

0.391 

7.8 

9.0 

-1.2 

0.423 

2.0 

1.8 

0.2 

0.799 


71.9 

71.6 

0.3 

0.915 

28.1 

28.4 

-0.3 

0.915 


16.1 

19.6 

-3.6 

0.270 

22.4 

21.6 

0.7 


12.4 

12.8 

-0.3 


49.1 

46.0 

3.1 


17.5 

21.1 

-3.6 

0.422 

25.4 

25.3 

0.1 


23.3 

22.6 

0.8 


33.8 

31.0 

2.7 



66.2 

69.8 

-3.6 

0.259 

33.8 

30.2 

3.6 

0.259 

2.88 

2.93 

-0.04 

0.263 




0.737 

1.1 

1.1 

0.0 


11.5 

8.2 

3.3 


4.6 

5.2 

-0.6 


42.6 

45.6 

-3.0 


32.2 

31.9 

0.2 


8.0 

7.9 

0.1 



(continued) 
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Table 3.2 (continued) 



Case- 

Non-Case- 


P-Value for 


Managed 

Managed 

Estimated 

Estimated 

Survey Item 

Group 

Group 

Difference 

Difference 

How far do you think you will actually go in school? (%) 




0.838 

Some high school 

1.9 

1.3 

0.6 


Finish high school 

13.3 

13.6 

-0.3 


Some college or trade/technical school 

10.5 

11.9 

-1.4 


Finish college or trade/technical school 

35.5 

38.3 

-2.8 


Graduate school after college 

26.4 

25.1 

1.3 


Don’t know 

12.5 

9.8 

2.7 


Caring adult at home (l-4) e 

3.38 

3.40 

-0.02 

0.403 

Caring adult at school (1-4) 

3.29 

3.25 

0.04 

0.328 

Caring adult outside of home or school (1-4) E 

3.45 

3.49 

-0.04 

0.206 

Joint test of difference between groups’ 1 

(X 2 = 40.1) 



0.718 

Number of students (total = 1,416) 

701 

715 




SOURCE: MDRC calculations based on the fall 2012 baseline student survey. 

NOTES: The analyses reported in this table are based on the stable student survey sample, which includes all 
students who responded to the spring 2013 and spring 2014 follow-up student surveys. Questions with the 
same wording as those in the baseline student survey appear in the follow-up student survey in Appendix D. 

The estimated differences between the case-managed group and the non-case-managed group are 
regression- 

adjusted using ordinary least squares, controlling for random assignment blocks by school. The values in the 
column labeled “Case-Managed Group” are the observed means for students randomly assigned to the case- 
managed group. The “Non-Case-Managed Group” values are the regression-adjusted means for students 
randomly assigned to the non-case-managed group, using the observed distribution of the case-managed 
group across random assignment blocks as the basis for the adjustment. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between the case-managed and non-case-managed groups. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Due to missing values, the number of students included varies by survey item. The sample size reported 
here is for the stable student survey sample. The percentage of missing data for any given survey item 
averages to 18 percent and ranges from 13 percent to 44 percent. 

“These survey categories are not mutually exclusive. 

b Of those who responded that English was not the predominant language at home, 92 percent reported 
speaking Spanish at home. 

Respondents without siblings old enough for high school are omitted. 

d Scale based on responses to survey questions 9a-9e, ranging from 1 = “not at all true” to 4 = “very much 
true.” Cronbach’s alpha = 0.83. 

e Scale based on responses to survey questions 17a-17g, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.81. 

f Scale based on responses to survey questions 8a-8f, ranging from 1 = “not at all true” to 4 = “very much 
true.” Cronbach's alpha = 0.89. 

g Scale based on responses to survey questions 15a-15f, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.89. 

h A chi-square test was used to determine whether there is a systematic difference between the case- 
managed group and the non-case-managed group when they joined the study, based on the characteristics 
included in this table as well as indicators of missing data for all relevant student characteristics. 
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any type of Level 2 service, that service should be documented in the system by a site coordina¬ 
tor. As described in Chapter 2, activities are recorded in the system as Level 2 services when 
those services are provided directly by CIS and when site coordinators connect case-managed 
students to services provided by someone else. 

CIS affiliates’ management infomiation systems provide infonnation about the services 
and other forms of support students received in up to eight categories: academics, behavior, 
attendance, social or life skills, resources to meet basic needs, college and career preparation, 
enrichment or motivation, and family (see Box 3.2 for examples of each). For every case- 
managed student, site coordinators track both “service contacts” — each time any service is 
received — and the length of time each service contact lasts. For example, if a student attended 
two tutoring sessions, those would be counted as two service “contacts” and the number of 
minutes each session lasted would also be recorded (rounded to the nearest 15-minute interval). 

The students who could have received services in this second year of the study were 
those who were originally assigned to the case-managed group during the 2012-2013 school 
year and who remained in their schools or moved to other schools with CIS programs during the 
2013-2014 school year. Among students in the stable student survey sample, 701 students were 
eligible to receive services in the second year of the study. Of these 701 students, 81 percent 
(566 students) had at least one service contact during the school year, as indicated in Table 3.3. 5 
These students received an average of nearly 20 service contacts during the year and an average 
of 18.4 total service hours, which amounts to approximately an hour of service about every 
other week during the school year. In the second year of the study, case-managed students had 
slightly more service contacts and hours than did case-managed students in the first year of the 
study (16.5 and 16.9, respectively). 6 Tables for the first year of the study are included in 
Appendix C. 

Overall, the type of service received by the greatest proportion of case-managed stu¬ 
dents was academic services: 73 percent of students received an average of 5.6 service contacts 
totaling 5.2 hours of service during the year. At least half of all students received services 
related to social and life skills, enrichment and motivation, and family, or received resources to 
help meet their basic needs. Just over 40 percent of students received college and career 


5 While reaching four out of five students is a notable accomplishment, the remaining 19 percent of the 
students assigned to the case-managed group did not have any direct services documented. They also had not 
dropped out of school, as they were present to take the student survey in the spring of the second study year. 

6 This increase may be related to the fact that, during the second year of the study, site coordinators could 
provide services from the start of the school year, rather than waiting until after random assignment was 
completed in the fall. It also may be related to the fact that they served a lower percentage of students assigned 
to the case-managed group in the second year (81 percent compared with 91 percent in the first year). 


48 



Box 3.2 

What Is Included in Each Service Type? 

Academic services: Adult or peer tutoring, homework assistance, study-skills activities, 
student/teacher conferences 

Behavioral services: Conflict-resolution groups, anger management or other behavioral 
counseling, violence-prevention activities, behavior monitoring and interventions 

Attendance services: In-person attendance check-ins and planning 

Social services/life skills: Goal-setting activities; self-esteem enhancement activities; 
girls/boys groups; social, relationship, and communication activities; team-building games and 
activities; crisis/grief counseling services 

Resources to meet basic needs: School supplies assistance; assistance with utilities, rent, etc.; 
food and clothing assistance; health activities/check-ups 

College/career preparation: College-admissions preparation and assistance, career counsel¬ 
ing, college visits and career field trips, college awareness activities/programs 

Enrichment/motivation: Community service, noncollege/career field hips, sports or exercise 
activities, scouting activities, arts and crafts activities, student recognition activities and 
incentives 

Family-related: Parent education, home visits, parent conferences and contacts, parent/family 
events and activities, family counseling 


preparation services, and 35 percent received behavioral services. Of the various service types 
students received, behavioral and social/life-skills services had the greatest number of service 
contacts — nearly 8 average contacts for each — and also had relatively high numbers of hours 
— 4.4 and 6.0 respectively. This infonnation confonns with site coordinators’ reports that 
students who participate in groups focused on behavior or social skills often do so on a regular 
basis for a portion of the school year (for example, one quarter or one semester). 7 


7 While Table 3.3 indicates that the smallest proportion of students received attendance services (approxi¬ 
mately 13 percent), data on this service type were only available for 62 percent of the students in the sample. 
The other service types had higher rates of data availability — all data sources provided information on 
academic, behavioral, and social/life-skills services. Data on resources to meet basic needs and on services 
related to college and career preparation, enrichment and motivation, and family were available for 84 percent 
of students in the sample. 
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Table 3.3 


Services Received by Case-Managed Students 
According to CIS Databases 


Service Received 

Percentage 
of Students 
Receiving Service 

Average Number of 
Times Service 

Was Received 

Average 
Total Hours of 

Service Received 

Overall 

Any service type 

80.7 

NA 

NA 

Across all service types 

NA 

19.9 

18.4 

By service type 

Academic 

72.6 

5.6 

5.2 

Behavioral 

34.5 

7.6 

4.4 

Social/life skills 

55.9 

7.7 

6.0 

Resources to meet basic needs 

54.2 

2.4 

2.0 

College/career preparation 

41.5 

2.0 

3.5 

Enrichment/motivation 

50.5 

2.7 

4.0 

Family-related 

52.2 

1.0 

0.5 

Attendance 

12.8 

2.6 

0.6 

Number of students 



701 


SOURCES: The Communities In Schools Data Management system (CISDM) and two local CIS 
service-provision databases (2013-2014). 

NOTES: The analyses in this table are for case-managed students only, and are based on the stable 
student survey sample, which includes all students who responded to the spring 2013 and spring 
2014 follow-up student surveys. 

NA = not applicable. 

The services offered are not mutually exclusive; a student could have received more than one 
type of service during his or her enrolled period. 

Calculations for the percentage of students receiving a given service are based on a consistent 
denominator of 701 case-managed students. However, not all service types were available in each 
data source. All data sources provided information on academic, behavioral, and social/life-skills 
services. Information on resources to meet basic needs and college/career preparation, 
enrichment/motivation, and family-related services was available for 83.6 percent of student 
records. Information on attendance services was available for 62.2 percent of student records. 

Rounding may cause slight discrepancies in averages and percentages. 

Services described in this table were received over the entire time students were enrolled in 
CIS case management during the 2013-2014 school year. The averages presented for number of 
times and total hours of service include only those students who received the service in question. 

Of all case-managed group students (701), 19.3 percent did not receive case management. In 
addition, of all non-case-managed group students (715), 7.7 percent received case management 
but are not included in this table. 
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Services Reported by Case-Managed and Non-Case-Managed 
Students 

To understand the difference that CIS case management makes for students, the research team 
also compared the types of services and activities reported by case-managed and non-case- 
managed students. The study students in the non-case-managed group experienced a “business- 
as-usual” condition — that is, they had access to the services that exist in their schools, includ¬ 
ing the Level 1 services provided or arranged by CIS. While schools did not consistently collect 
data on the services received by non-case-managed students, the research team did administer 
surveys asking case-managed and non-case-managed students about the activities they partici¬ 
pated in and the services they received. 

Table 3.4 shows case-managed and non-case-managed students’ reports of service par¬ 
ticipation. For almost every service category shown in the table, higher percentages of case- 
managed students received services than non-case-managed students, whether the difference 
between them is statistically significant or not — as was also the case in the first year of the 
study. Significantly greater proportions of case-managed students reported receiving tutoring or 
homework help; meeting with mentors; and earning rewards for positive behavior, grades, or 
reaching goals. For example, 39 percent of the students in the case-managed group participated 
in mentoring, compared with 27 percent of students in the non-case-managed group. In addi¬ 
tion, larger proportions of students in the case-managed group reported having in-school 
meetings with adults, including individual and group meetings focused on academics, individu¬ 
al meetings to receive support during life-changing events, and group meetings for social 
activities. 

Students in the case-managed group were also more likely than non-case-managed students to 
report meeting with an adult at school to set specific goals for the year (68 percent compared 
with 50 percent) and to report that an adult in school connected them with support programs or 
help outside of school (51 percent compared with 40 percent). Much of this in formation 
comports with site coordinators’ survey responses about the types of services available for case- 
managed students in their schools (described in Chapter 2). 8 


8 Students were also asked about the frequency with which they participated in these activities during the 
school year. Their responses are shown in Appendix Table C.2. Case-managed students reported participating 
in many support activities a greater number of times per year than non-case-managed students, a pattern similar 
to the one shown in Table 3.4. 
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Table 3.4 


Effects on Percentages of Students Who Received Services 
According to the Student Survey 


Survey Item 

Case- 

Managed 

Group 

Non-Case- 

Managed Estimated 
Group Difference 

P-Value for 

Estimated 

Effect 

In-school support activities (%) 

Tutoring or homework help 

73.7 

67.9 

5.8 ** 

0.018 

Mentoring 

38.6 

27.2 

11.4 *** 

0.000 

Community service/volunteering 

36.8 

34.7 

2.1 

0.405 

Exercise class or club f 

12.8 

18.2 

4^ ^ sj{ sjs sjs 

0.006 

Positive behavior program 
such as drug-free/antibullying 

30.6 

28.9 

1.7 

0.474 

College planning activity 

50.9 

52.2 

-1.3 

0.635 

Career planning activity 

39.9 

38.5 

1.4 

0.583 

Job shadowing/internship 

21.1 

18.6 

2.5 

0.232 

After-school program 

36.0 

37.6 

-1.6 

0.532 

Assistance such as school supplies, food, bus 
pass, clothing, or gifts 

31.8 

28.2 

3.5 

0.146 

Health check-up 

52.0 

52.3 

-0.3 

0.910 

In-school meetings with adults (%) 

Individual meeting about academics f 

74.5 

64.2 

10.3 *** 

0.000 

Individual meeting for support during a 
life-changing event 

34.5 

30.2 

4.3 * 

0.095 

Individual meeting about personal 
goals and behavior 

50.6 

42.9 


0.005 

Group meeting about academics 

54.1 

42.9 

11 2 

0.000 

Group meeting for support during 
a life-changing event 

27.1 

26.8 

0.3 

0.915 

Group meeting about personal 
goals and behavior 

38.9 

34.5 

4.4 * 

0.093 

Group meeting for social activities 

50.1 

30.1 

20.0 *** 

0.000 


(continued) 
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Table 3.4 (continued) 


Survey Item 

Case 

Managed 

Group 

Non-Case 

Managed Estimated 
Group Difference 

P-Value for 

Estimated 

Effect 

Met with an adult at school to set 
specific goals for the year (%) 

67.8 

49.9 

j >1 g *** 

0.000 

Could earn rewards for improving grades, 
attendance, behavior, or for reaching 
or making progress toward goals (%) 

59.4 

53.6 

5.7 ** 

0.029 

An adult in school connected student to support 
programs or help outside of school (%) 

50.9 

39.8 

Yy y *** 

0.000 

Number of students (total = 1,416) 

701 

715 




SOURCE: MDRC calculations based on the spring 2014 follow-up student survey. 

NOTES: The analyses reported in this table are based on the stable student survey sample, which 
includes all students who responded to the spring 2013 and spring 2014 follow-up student surveys. 

The estimated differences between the case-managed group and the non-case-managed group are 
regression-adjusted, controlling for the blocking of random assignment by school, as well as the 
following baseline characteristics: race, gender, free or reduced-price lunch status, English as a second 
language, whether qualified for a gifted program, and special education status. 

The values in the column labeled “Case-Managed Group” are the observed means for students 
randomly assigned to the case managed group. The “Non-Case-Managed Group” values in the next 
column are the regression-adjusted means for students randomly assigned to the non-case-managed 
group, using the observed mean covariate values for the case-managed group as the basis for the 
adjustment. 

A two-tailed t-test was applied to differences between the case-managed and non-case-managed 
groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The sample size reported in the table is for the stable student survey sample. However, the sample 
size varies across outcomes due to missing data. The percentage of missing data for any given outcome 
does not exceed 4 percent. 

A dagger (f) indicates that there is a statistically significant difference between the estimated effects 
in Year 1 and Year 2 at the 5 percent level. 

Rounding may cause slight discrepancies in calculating sums and differences. 


Services Received by High- and Moderate-Risk Case-Managed 
Students 

While the primary analyses for this study examine all study students, both CIS and school staff 
members noted that the students eligible for case management demonstrate varied levels of 
need. As noted in the interim report, CIS staff members explained that certain students have 
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more intensive needs — which are evidenced by failing grades, very poor attendance, behavior¬ 
al infractions, and other social and personal challenges — and that these students should 
therefore receive more services than other case-managed students. To begin to understand how 
service provision and receipt may vary based on levels of student need, the research team 
conducted a set of analyses to examine variation in service receipt for students who could be 
considered at “high risk” of dropping out compared with more “moderate-risk” students. 9 Using 
school records data, the research team classified students as being at high risk if they were 
chronically absent (had less than 90 percent attendance), failed one or more core courses, or 
were suspended in the 2011-2012 school year — the year before the study. 10 

The first report from this study found that there was little difference in the services re¬ 
ceived by high-risk and moderate-risk students, though it did find that a significantly greater 
proportion of the high-risk students received any services (93 percent compared with 88 percent 
among moderate-risk students) and it found that more high-risk students received behavioral 
services, but fewer received college and career-preparation and enrichment services. 11 Table 3.5 
details the CIS services received by case-managed students during the second year of the study 
and shows that there were many differences in the services received by high- and moderate-risk 
students — more than there had been in the first year of the study. In nearly all of these cases, 
more moderate-risk students than high-risk students received services in Year 2, and received 
them more often. 

Overall, 74 percent of the high-risk case-managed students received services in the sec¬ 
ond year of the study, which means that one out of four case-managed students in the high-risk 
subgroup did not receive any. This number represents a substantial decrease from the first year 
of the study, when 93 percent of high-risk case-managed students received services. While the 
proportion of moderate-risk case-managed students receiving services remained steady, the 
proportion of high-risk students receiving services dropped below the proportion of moderate- 
risk students in Year 2. 

The bottom panel of Table 3.5 shows the percentages of high- and moderate-risk stu¬ 
dents who received specific services. There were significant differences between the groups in 
the percentages of students receiving every type of service, with the differences in all but 
behavioral services favoring the moderate-risk students. Compared with the first year, there 
were decreases in the percentages of high-risk students receiving every service type, with 


9 The two groups are referred to as high and moderate risk rather than high and low risk because being 
eligible for case management itself implies some level of risk. 

10 The research team also separately examined service differentiation based on each of these risk factors, 
but the results were not substantially different from the results for the group as a whole. 

u Corrin et al. (2015). 
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Table 3.5 


Services Received by High- and Moderate-Risk Case-Managed Students 

According to CIS Databases 



Percentage of Students 

Receiving Service 

Average Number of 

Times Service Was Received 

Average Total Flours 
of Service Received 

Service Received 

Fligh-Risk 

Students 

Moderate- 

Risk Students 

Fligh-Risk 

Students 

Moderate- 

Risk Students 

Fligh-Risk 

Students 

Moderate- 

Risk Students 

Overall 

Any service type 

74.4 

9 *** 

NA 

NA 

NA 

NA 

Across all service types 

NA 

NA 

19.8 

20.0 

19.1 

18.1 

By service tvoe 

Academic 

68.2 

75.5 ** 

6.3 

5.1 ** 

6.2 

4.6 ** 

Behavioral 

39.0 

31.6 ** 

7.0 

8.1 

4.1 

4.5 

Social/life skills 

48.4 

60.9 *** 

7.4 

7.9 

6.3 

5.8 

Resources to meet basic needs 

44.4 

60.6 *** 

2.3 

2.5 

2.0 

2.0 

College/career preparation 

35.0 

45.8 *** 

1.7 

2.1 * 

3.5 

3.5 

Enrichment/motivation 

40.1 

57 3 *** 

2.4 

2.8 

3.6 

4.2 

Family-related 

45.9 

56.4 *** 

1.0 

1.0 

0.5 

0.5 

Attendance 

10.1 

14.6 * 

2.6 

2.7 

0.4 

0 7** 

Number of students 

277 

424 

277 

424 

277 

424 


(continued) 



Table 3.5 (continued) 


SOURCES: The Communities In Schools Data Management system (CISDM) and two local CIS 
service-provision databases (2013-2014). 

NOTES: The analyses in this table are for case-managed students only, and are based on the stable 
student survey sample, which includes all students who responded to the spring 2013 and spring 2014 
follow-up student surveys. 

The services offered are not mutually exclusive; a student could have received more than one type of 
service during his or her enrolled period. 

Calculations for the percentage of students receiving a given service are based on a consistent 
denominator of 277 high-risk students and 424 moderate-risk students. However, not all service types 
were available in each data source. All data sources provided information on academic, behavioral, and 
social/life-skills services. Information on resources to meet basic needs and college/career preparation, 
enrichment/motivation, and family-related services was available for 83.6 percent of student records. 
Information on attendance services was available for 62.2 percent of student records. 

Rounding may cause slight discrepancies in averages and percentages. 

A two-tailed t-test was conducted to test for differences between findings for high- and moderate-risk 
students. Statistical significance levels are indicated as: *** = 1 percent, ** = 5 percent, * = 10 percent. 

Services described in this table were received over the entire time students were enrolled in CIS case 
management during the 2013-2014 school year. The averages presented for number of times and total 
hours of service include only those students who received the service in question. 

High-risk students are defined as those who were chronically absent, who failed a core course, or who 
were ever suspended in the 2011-2012 school year. Moderate-risk students include those who were never 
chronically absent, never failed a core course, and were never suspended in the 2011-2012 school year. 
Students who were missing data on chronic absenteeism, suspension, and course failure were classified 
as moderate-risk. 

Of all high-risk case-managed group students (277), 25.6 percent did not receive case management. In 
addition, of all high-risk non-case-managed group students (259), 7.7 percent received case management 
but are not included in this table. 

Of all moderate-risk case-managed group students (424), 15.1 percent did not receive case 
management. In addition, of all moderate-risk non-case-managed group students (456), 7.7 percent 
received case management but are not included in this table. 


double-digits drops in services related to academics, behavior, and social and life skills, and in 
resources to meet basic needs. 12 While there were also decreases in the percentages of moder¬ 
ate-risk students receiving academic and behavioral services, the declines were not as steep, and 
there were small increases in the percentages of moderate-risk students receiving social/life- 
skills, college and career preparation, enrichment, and family-related services. 

Among the high- and moderate-risk students who did receive a given type of service, 
however, there were fewer differences between the two subgroups in the amount of services 
they received, which more closely mirrors the findings from the first year. One notable excep- 


12 Corrin et al. (2015). 
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tion is that high-risk students who received academic services during the second year of the 
study received more service contacts and hours, on average, than moderate-risk students 
receiving this service. Specifically, high-risk students getting academic support received an 
average of 6.3 service contacts totaling 6.2 hours, compared with 5.1 contacts and 4.6 hours for 
moderate-risk students. 13 

While the services for high- and moderate-risk students were not as differentiated as 
one might expect, the research team explored the possibility that services might have been 
differentiated in other ways. Specifically, the team analyzed whether services varied based on 
whether students faced substantial struggles in their academic performance, behavior, or 
attendance. For example, the team examined whether students who struggled academically 
received substantially greater amounts of academic services. It turned out that services were 
largely not differentiated based on these indicators. It is possible that other factors played a role 
in creating differences in service receipt (for example, whether a student was experiencing 
emotional challenges or other personal difficulties), but the research team was not able to 
measure such factors in this study. 


Discussion 

This chapter includes information about the students participating in this study and describes the 
services received by students assigned to both the case-managed and non-case-managed groups. 
Greater proportions of students in the case-managed group reported participating in support 
activities and services than those in the non-case-managed group. However, it is critical to 
determine whether case-managed students received enough additional services to lead to 
differences in their school outcomes. This question will be explored further in Chapter 4. 

This chapter also explored whether high-risk and moderate-risk students received dif¬ 
ferent amounts of services. It appears that CIS may be providing more services to those who are 
easier to serve — moderate-risk students — and who might fare relatively well even in the 
absence of those services. Site coordinators appeared to struggle to serve the students who were 
probably at the greatest risk of not succeeding in school. While site coordinators reached 91 
percent of the high-risk case-managed students during the first year of the study, they were only 
able to reach 74 percent during the second year. These results suggest that CIS may benefit 
from developing strategies for serving or continuing to serve higher-risk students. These 


l3 In addition to the analysis presented in this section, the research team also examined the survey respons¬ 
es of case-managed and non-case-managed students based on their status as being at either high or moderate 
risk. There is some variation in activities undertaken by high- and moderate-risk case-managed and non-case- 
managed students, but there is no consistent pattern of effects. High-risk students are not consistently doing 
more activities or receiving more services. See Appendix Table C.3. 
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students may be the most challenging to serve or to keep attached to support from year to year, 
but they are also the students who need support the most. 

Finally, while the case management process includes an individual needs assessment, 
when the research team analyzed whether students who were struggling with academic perfor¬ 
mance, behavior, or attendance received different services in response to those needs, it largely 
found that they did not. CIS may therefore want to consider how to target services more 
specifically, to make sure that students receive services that match their needs. 
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Chapter 4 

The Effect of Case Management 


This chapter focuses on the effect of Communities In Schools (CIS) Level 2 case management 
on students’ academic and behavioral outcomes after two years. As discussed in Chapter 1, this 
study uses a random assignment research design, estimating the effect of Level 2 case manage¬ 
ment by comparing the outcomes of students who were randomly assigned to be offered case 
management (the case-managed group) with the outcomes of students who were not, but who 
retained access to whatever other types of services were available in their schools (the non-case- 
managed group). Students in the case-managed group were offered Level 2 services for two 
school years, and this chapter describes the effect of these services on student outcomes in the 
second follow-up year. As discussed in the previous chapter, however, not all students in the 
case-managed group actually participated in Level 2 case management for a second year, so the 
findings in this chapter represent the effect of offering students two years of case management 
rather than the effect of receiving two years of case management. 1 

As detailed in Chapter 1, this study investigates the effect of CIS case management on 
two types of outcomes: nonacademic mediating outcomes and more traditional school out¬ 
comes. The nonacademic outcomes are behaviors and attitudes believed to be precursors to 
students’ improved success in school: school engagement, students’ relationships with adults 
and peers, students’ self-perception, and students’ educational aspirations and expectations 
(shown in Figure 1.1 as “Mediating Factors”). Examining effects in these areas can provide 
insight into how CIS may produce effects on traditional school outcomes (shown in Figure 1.1 
as “Outcomes”). Among the school outcomes, this study designated chronic absenteeism and 
course failure as the main or central outcomes to track, because chronic absenteeism and course 
failure are among the factors that most strongly predict whether students stay in school until 
graduation (CIS’s ultimate goal). 2 In addition, the study is tracking measures of school progress 
(core credit accumulation), student behavior (average attendance rate and number of suspen¬ 
sions), and academic achievement (average course grades). 

The tables in this chapter show estimated effects on each of these outcomes in the sec¬ 
ond follow-up year. The tables also present these estimates as effect sizes, a metric that makes it 


'in other words, the findings in this report are "intent-to-treat” estimates of the effect of offering students 
two years of services. 

2 Given the time frame of this evaluation, the study team cannot track the full sample of middle school and 
high school students through high school graduation. 
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possible to compare the size of effects in different areas. (See Box 3.1 for an explanation of how 
to interpret the findings in the report tables.) 3 Highlights from the chapter include: 

• Nonacademic mediating outcomes: In the second follow-up year, CIS case 
management had positive and statistically significant effects on students’ re¬ 
ports of having a caring adult at home, at school, and outside of home and 
school, as well as on the quality of their peer relationships. In addition, case 
management also had positive and statistically significant effects on students’ 
engagement with school, their educational attitudes, and their belief that edu¬ 
cation has a positive value for their lives. There was no difference between 
case-managed and non-case-managed students for the remaining mediating 
outcomes — school- and non-school-sponsored extracurricular activities and 
educational goals and expectations. 

• Traditional school outcomes: In the second follow-up year, CIS case man¬ 
agement did not have statistically significant positive effects on the students’ 
school outcomes. Students in the case-managed and non-case-managed 
groups had similar rates of chronic absenteeism and attendance, core course 
failure, credit accumulation, and course marks. However, case-managed stu¬ 
dents had more suspensions, on average, than non-case-managed students 
and this difference was statistically significant. Thus, it cannot be concluded 
that CIS case management improved students’ attendance, course perfor¬ 
mance, or behavior. 


Effects on Nonacademic Mediating Outcomes 

As shown in the case management logic model (Chapter 1), nonacademic mediating 
outcomes are considered predictors of future academic outcomes. It is expected that if students 
experience positive effects in some or all of these areas, they will be more likely to experience 
effects on subsequent school outcomes that are directly related to reducing student dropouts 
(CIS’s main mission). As discussed in the previous chapter, case-managed students receive 
eight types of direct services, and the most frequently offered services are those in the areas of 
academics, social or life skills, resources to meet basic needs, and enrichment or motivation. 
According to the CIS model, being provided with these services should increase the level of 
attention a student receives from a CIS site coordinator, through connections with other adults 
within and outside the school, or both. These mechanisms of additional support should help 
students feel more engaged, motivated, and connected in school. 


’Estimated effects after one year of case management can be found in Appendix C. 
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Table 4.1 presents the estimated effects of CIS Level 2 case management on nonaca¬ 
demic mediating outcomes in the second year of the study. The outcomes in this table are drawn 
from the survey administered to students in the spring of 2014. The first panel of the table 
relates to students’ reports that they have relationships with caring adults in school, at home, or 
outside of school and home. 4 These three outcomes are based on scales rating whether students 
feel that the adults care, listen, believe in them, and encourage them. There are small, positive, 
statistically significant differences between the case-managed and non-case-managed groups for 
all three items related to reports of a caring adult relationship: caring adult at home (effect size = 
0.15, p-value = 0.001), caring adult at school (effect size = 0.14, p-value = 0.004), and caring 
adult outside of home or school (effect size = 0.14, p-value = 0.004). These differences indicate 
that after two years of Level 2 case management services, students in the case-managed group 
had more caring relationships at home, in school, and outside of home and school than non¬ 
case-managed-students. 5 CIS’s positive effect on relationships with caring adults outside of 
school or home is consistent with the finding that case-managed students reported more often 
that they have access to an adult in school who can connect them to support or help outside of 
school (Table 3.4). 

It worth noting that CIS’s effect on students’ relationships with adults is a preventive 
one. Compared with their answers when they enrolled in the study (Table 3.2), in the second 
year non-case-managed students were less likely to report that they had caring adults at home or 
outside of home and school, while the case-managed students were about equally likely to do 
so. Both case-managed and non-case-managed students were less likely to report relationships 
with caring adults at school in the second follow-up year (3.26 and 3.16 respectively) than they 
were when they enrolled in the study (3.29 and 3.25 respectively); however, case-managed 
students’ reports of having such relationships declined less than non-case-managed students’, 
resulting in a positive effect on the case-managed students. 

The next scale reported in Table 4.1 is based on student reports about their friends and 
peers. Items in this scale relate to having caring friends, friends who help out during hard times, 
friends who talk about problems, and friends who do what is right. The case-managed group 
reported a higher quality of friendships than the non-case-managed group, and the estimated 
effect is statistically significant (effect size = 0.15; p-value = 0.002). This result suggests that 


^Examples of such adults outside of school and home could include a counselor at a family counseling 
center, a coach for a sports team not affiliated with the school, a staff member from a community organization 
such as a YMCA, or a youth minister at a local church’s after-school program. 

5 After one year of services, there was a significant and positive effect on students’ reports of having a car¬ 
ing adult outside of home or school and not on the other outcomes related to caring adults (see Appendix Table 
C.4) 


61 



Table 4.1 


Effects on Students’ Attitudes and Activities 


Survey Item 

Case- 

Managed 

Group 

Non-Case- 

Managed Estimated 
Group Effect 

Effect 

Size 

P-Value for 

Estimated 

Effect 

Caring adult at home (l-4) a 

3.42 

3.32 

0.10 

0.15 

*** 

0.001 

Caring adult at school (l-4) b 

3.26 

3.16 

0.10 

0.14 

*** 

0.004 

Caring adult outside of home or school (1 -4)’ 

3.46 

3.34 

0.12 

0.14 

*** 

0.004 

Friend quality (l-4) d 

2.99 

2.90 

0.09 

0.15 

*** 

0.002 

School-sponsored extracurricular activities 







Students selecting at least 1 activity (%) 

81.7 

85.0 

-3.2 

-0.09 


0.107 

Mean number of activities done sometimes 

2.58 

2.7 

-0.16 

-0.05 


0.379 

Mean number of activities done often 

2.74 

2.7 

0.06 

0.02 


0.741 

Non-school-sponsored extracurricular activities 







Students selecting at least 1 activity (%) f 

74.9 

75.6 

-0.8 

-0.02 


0.742 

Mean number of activities done sometimes 

1.64 

1.6 

0.02 

0.01 


0.836 

Mean number of activities done often 

1.55 

1.6 

-0.09 

-0.04 


0.390 

Student engagement with school (l-4) e 

2.79 

2.70 

0.09 

0.11 

** 

0.020 

f 

Educational attitudes (1-4) 

3.02 

2.97 

0.05 

0.09 

** 

0.037 

Positive educational self-perception and effort® 

2.70 

2.66 

0.04 

0.05 


0.263 

Negative educational self-perception and effort 1 ’ 

2.11 

2.07 

0.03 

0.05 


0.323 

Positive valuation of education 1 

3.31 

3.21 

0.11 

0.15 

*** 

0.001 

How far would you like to go in school? (%) f 






0.412 

Some high school 

0.9 

0.8 

0.1 

0.01 



Finish high school 

7.7 

9.5 

-1.7 

-0.06 



Some college or trade/technical school 

3.7 

5.7 

-2.0 

-0.09 



Finish college or trade/technical school 

33.8 

33.3 

0.4 

0.01 



Graduate school after college 

42.9 

41.4 

1.5 

0.03 



Don’t know 

11.0 

9.6 

1.5 

0.05 



How far do you think you will actually go in school? (%) 





0.477 

Some high school 

1.0 

2.1 

-1.1 

-0.08 



Finish high school 

13.2 

13.4 

-0.3 

-0.01 



Some college or trade/technical school 

9.3 

9.7 

-0.4 

-0.01 



Finish college or trade/technical school 

32.6 

33.9 

-1.2 

-0.03 



Graduate school after college 

29.5 

28.5 

1.0 

0.02 



Don’t know 

14.4 

12.8 

1.6 

0.05 



Number of students (total =1,416) 

701 

715 






(continued) 
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Table 4.1 (continued) 


SOURCE: MDRC calculations based on the spring 2014 follow-up student survey. 

NOTES: The analyses reported in this table are based on the stable student survey sample, which includes all 
students who responded to the spring 2013 and spring 2014 follow-up student surveys. See Appendix D for 
survey questions. 

Estimated effects are regression-adjusted using ordinary least squares, controlling for random assignment 
blocks by school, as well as the following baseline characteristics: race, gender, free or reduced-price lunch 
status, English as a second language, whether qualified for a gifted program, special education status, and a 
baseline measure of the outcome variable. 

The values in the column labeled “Case-Managed Group” are the observed means for students randomly 
assigned to the case-managed group. The “Non-Case-Managed Group” values are the regression-adjusted 
means for students randomly assigned to the non-case-managed group, using the observed mean covariate 
values for the case-managed group as the basis for the adjustment. 

Effect sizes are calculated by dividing the impact estimate by the standard deviation of the outcome 
measure for students in the stable student survey sample who are in the non-case-managed group. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between case-managed and non-case-managed groups. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The sample size reported in the table is for the stable student survey sample. However, the sample size 
varies across outcomes due to missing data. The percentage of missing data for any given outcome does not 
exceed 1 percent. 

A dagger (f) indicates that there is a statistically significant difference between the estimated effects in 
Year 1 and Year 2 at the 5 percent level. 

a Scale based on responses to survey questions 17a-17g, ranging from 1 = "not at all true” to 4 = “very 
much true.” Cronbach’s alpha = 0.90. 

b Scale based on responses to survey questions 8a-8f, ranging from 1 = “not at all true” to 4 = “very much 
true.” Cronbach’s alpha = 0.89. 

c Scale based on responses to survey questions 15a-15f, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach’s alpha = 0.94. 

d Subscale based on responses to survey questions lOa-lOf, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.72. 

e Scale based on responses to survey questions 9a-9e, ranging from 1 = “not at all true” to 4 = “very much 
true.” Cronbach’s alpha = 0.84. 

f Scale based on responses to survey questions 1 la-1 In, ranging from 1 = "not at all true” to 4 = “very 
much true.” Cronbach’s alpha = 0.83. 

g Subscale based on responses to survey questions 1 lb, lie, Ilf, 1 lg, ranging from 1 = "not at all true” to 4 
= “very much true.” Cronbach’s alpha = 0.71. 

h Subscale based on responses to survey questions 11a, 11c, lid, 1 lh, ranging from 1 = "not at all true” to 
4 = “very much true.” Cronbach's alpha = 0.75. 

'Subscale based on responses to survey questions 1 li-1 In, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.79. 


Level 2 case management may help the case-managed students form more trusting and support¬ 
ive relationships with their peers and friends than do non-case-managed students. This effect is 
consistent with the finding presented in Chapter 3 that case-managed students reported being 
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more involved in peer-group meetings led by adults, a setting in which supportive peer relation¬ 
ships could have been cultivated. 

Students also responded to survey questions about how often they participated in 
school-sponsored and non-school-sponsored extracurricular activities such as school sports 
teams, art or music groups, academic clubs, Junior ROTC, or cultural groups. The differences in 
the percentages of case-managed and non-case-managed students who participated in at least 
one school-sponsored activity or in at least one non-school-sponsored activity were less than 2 
percentage points in each case and not statistically significant. In addition, case-managed and 
non-case-managed students who did participate in activities did so with similar frequency. 

The items in the school-engagement scale shown in Table 4.1 ask whether students feel 
happy, feel safe, feel that they are treated fairly, and feel like a part of their school. The items in 
the educational-attitudes scale ask about students’ own perceptions of whether they do well at 
school, plan their work, persist with homework and schoolwork, give up easily, or have trouble 
figuring out answers in school. There are statistically significant differences between case- 
managed students’ and non-case-managed students’ reports about their engagement with school 
(effect size = 0.11, p-value = 0.020) and overall educational attitudes (effect size = 0.09, p-value 
= 0.037), meaning that case-managed students reported greater engagement with school and 
more positive educational attitudes than did non-case-managed students. 6 It is interesting to note 
that both case-managed and non-case-managed students’ ratings on the school engagement 
scale decreased from the baseline year (when it was approximately 2.90 for both groups), but 
decreased less for the case-managed group (from 2.88 to 2.79) than the non-case-managed 
group (from 2.93 to 2.70). 

Table 4.1 shows three subscales related to students’ educational attitudes. The first sub¬ 
scale focuses on students’ positive educational self-perception and effort. The items in this scale 
ask students whether they believe they do well at school, feel that they are as smart as other 
students, and are persistent with homework and study plans. The second subscale relates to 
negative educational self-perceptions: for example, whether students think they are slow in 
finishing their homework, have trouble figuring out answers, or easily give up if a task is hard. 
There is no notable or statistically significant difference between case-managed and non-case- 
managed groups in their positive and negative educational self-perceptions. The third educa¬ 
tional-attitude subscale is about students’ view of education, including items about whether 
students believe that education will be valuable to get a job, whether it is important to get good 
grades, and whether school is useful in making good decisions in life. For this subscale there is 
a statistically significant difference between the case-managed and non-case-managed groups 

6 After one year of services, there was a positive effect on student engagement with school, but the result 
was not statistically significant (see Appendix Table C.4). 
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(effect size = 0.15; p-value = 0.001), which suggests that compared with non-case-managed 
students, the support that case-managed students receive through CIS may be helping them 
value education more and understand how it can influence their future. 7 

Finally, Table 4.1 shows the effect of CIS Level 2 case-managed services on students’ 
academic goals and expectations. An omnibus test of statistical significance in this area shows 
that there is no difference overall between the case-managed and non-case-managed group with 
regard to their reported educational goals — how far they would like to go in their education — 
and educational expectations — how far they think they actually will go with their education. 
While case-managed students see greater value in their education than their non-case-managed 
peers, this perception has not improved their self-reported educational goals and expectations. 


Effects on Academic Outcomes 

Table 4.2 presents the estimated effect of CIS Level 2 case management on traditional school 
outcomes after two years of offering services. As stated at the beginning of this chapter, the 
study’s main outcomes are related to factors that impede students’ progress toward graduation: 
chronic absenteeism and course failure. The study also tracked additional outcomes related to 
student behavior and other aspects of course performance and attendance, which also predict 
students’ progress toward graduation. It is worth noting that in Table 4.2, the desired effect may 
be either positive or negative — that is, for chronic absenteeism, course failure, and average 
number of suspensions, a negative estimated effect would indicate fewer of these problems for 
case-managed students than for non-case-managed students, and therefore better outcomes for 
the case-managed group. 

The results in Table 4.2 show that in the second follow-up year, there is no difference 
between case-managed and non-case-managed students in their rates of chronic absenteeism or 
in the percentages of them who failed at least one core course. 8 Compared with baseline levels 
(Table 3.1), the rate of chronic absenteeism increased among both case-managed and non-case- 
managed students (from around 9 percent to 21 percent). A similar pattern is seen for the 
percentage of students who failed at least one core course (an increase from around 22 percent 
to 32 percent among both groups of students). 

Among the additional outcomes the study tracked, the one school-progress outcome — 
core credit accumulation toward graduation — reflects the percentage of core credits 


7 The result after one year of services was similar. 

8 This result is similar to the estimated effect after one year of services (see Appendix Table C.5). 
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Table 4.2 


Effects on Students’ Academic and Behavioral Outcomes 


Outcome 

Case- 

Managed 

Group 

Non-Case- 

Managed Estimated 
Group Effect 

Effect 

Size 

P-Value for 

Estimated 

Effect 

Main outcomes 

Chronic absenteeism 3 (%) 

21.3 

21.4 

-0.1 

0.00 

0.973 

Failed at least 1 core course (%) 

32.2 

31.6 

0.6 

0.01 

0.780 

School progress 

Core credit accumulation for graduation 11 (%) 

40.7 

41.5 

-0.8 

-0.06 

0.464 

Behavior 

Average attendance rate (%) 

92.6 

92.3 

0.3 

0.04 

0.422 

Number of suspensions 

1.55 

1.22 

0.33 

0.11 * 

0.051 

Academic achievement 

Average core course marks (%) 

79.1 

79.3 

-0.3 

-0.03 

0.483 

Number of students (total = 1,501) 

751 

750 





SOURCE: MDRC calculations based on student records obtained from school districts. 

NOTES: The analyses reported in this table are based on the stable school records sample, which includes 
all students with course-failure data for the 2012-2013 school year. 

Estimated effects are regression-adjusted using ordinary least squares, controlling for random 
assignment blocks by school, as well as the following baseline characteristics: race, gender, free or 
reduced-price lunch status, English as a second language, whether qualified for a gifted program, special 
education status, and a baseline measure of the outcome variable. 

The values in the column labeled “Case-Managed Group” are the observed means for students 
randomly assigned to the case-managed group. The “Non-Case-Managed Group” values are the 
regression-adjusted means for students randomly assigned to the non-case-managed group, using the 
observed mean covariate values for the case-managed group as the basis for the adjustment. 

Effect sizes are calculated by dividing the impact estimate by the standard deviation of the outcome 
measure for students in the school records analysis sample who are in the non-case-managed group. 
Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between case-managed and non-case-managed groups. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The sample size reported in the table is for the stable school records sample. However, the sample size 
varies across outcomes due to missing data. The percentage of missing data is 0.5 percent for attendance 
rates and absenteeism, 16 percent for credit accumulation, and 0 percent for other outcomes. 

a A student is considered chronically absent if he or she has an attendance rate below 90 percent. 
b This outcome is for high school students only, and in Year 2 is a cumulative measure for the 2012- 
2013 and 2013-2014 school years; the state-level graduation requirements for the four core subjects are 
used in this calculation. A student can be expected to accumulate about 25 percent of core credits in each 
year of high school. 
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accumulated to meet state graduation requirements. To graduate from high school in four years, 
a student would need to earn an average of 25 percent of required core credits annually. The 
results in Table 4.2 show that over two years the non-case managed group accumulated a 
marginally higher percentage of these credits (by around 1 percentage point), but this difference 
is not statistically significant. 

The student-behavior domain includes effects on average attendance and number of 
suspensions. The estimated difference between the average attendance rates of the case- 
managed and non-case managed groups is small and not statistically significant. (At baseline the 
average attendance rate was around 3 percentage points higher for both case-managed and non¬ 
case-managed students.) 9 However, there is a statistically significant effect on the number of 
suspensions: Case-managed students were suspended 0.33 times more than non-case-managed 
students (effect size = 0.11, p-value = 0.051). 10 Finally, both groups of students had similar 
academic achievement as measured by average course marks: 79 out of 100 points, or about a 
B-/C+ average mark. 11 

Effects on academic outcomes were also examined for the high- and moderate-risk sub¬ 
groups of students (defined in Chapter 3). These results are shown in Table 4.3. The findings 
are similar to the overall findings. Among the moderate-risk students there are no statistically 
significant differences between the case-managed and the non-case-managed group. Among the 
high-risk students there is a significant difference between the case-managed and non-case- 
managed group only when it comes to numbers of suspensions: The case-managed high-risk 
students had more suspensions than the non-case-managed high-risk students (effect size = 
0.25, p-value = 0.049). 

Finally, effects were also examined separately for students who were assigned to begin 
receiving CIS case management in sixth or ninth grade (versus other grade levels), because 
these are important transition years into middle or high school and also the grade levels when 
students typically join CIS site coordinators’ caseloads. 12 These analyses suggest that CIS case 
management may have greater potential to improve the outcomes of students who join it in 


"’Attendance rate is calculated as the number of days present divided by the total number of days in the 
school year. A student is chronically absent if his or her attendance rate is below 90 percent. While the 
attendance rate does include chronically absent students, it may mask the number of students who have 
attendance problems since they are a small proportion of the sample. The chronic absenteeism measure 
describes a subset of students who are not regularly attending school. 

10 Afler one year of services the effect on number of suspensions was small and not statistically significant 
(see Appendix Table C.5). 

"The average course marks were almost 2 percentage points higher for both groups of students in the 
baseline year. 

12 See Appendix Tables C.6 and C.7 for the full results of these analyses. 
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Table 4.3 


Effects on Academic and Behavioral Outcomes 
Among High- and Moderate-Risk Students 



Case- 

Non-Case- 



P-Value for 


Managed 

Managed 

Estimated 

Effect 

Estimated 

Outcome 

Group 

Group 

Effect 

Size 

Effect 

High-risk students 

Main outcomes 






Chronic absenteeism 11 (%) 

35.3 

33.4 

1.9 

0.05 

0.610 

Failed at least one core course (%) 

48.7 

46.6 

2.1 

0.05 

0.597 

School progress 






Core credit accumulation for graduation (%) 

18.4 

18.9 

-0.5 

-0.05 

0.735 

Behavior 






Average attendance rate (%) 

89.4 

89.2 

0.1 

0.01 

0.870 

Number of suspensions 

2.98 

2.23 

0.75 ** 

0.25 

0.049 

Academic achievement 






Average core course marks (%) 

75.5 

75.8 

-0.3 

-0.04 

0.668 

Number of students (total = 588) 

306 

282 




Moderate-risk students 






Main outcomes 






Chronic absenteeism 11 (%) 

11.4 

13.3 

-1.9 

-0.05 

0.376 

Failed at least one core course (%) 

20.8 

21.6 

-0.8 

-0.02 

0.760 

School progress 






Core credit accumulation for graduation (%) 

20.5 

21.1 

-0.6 

-0.07 

0.481 

Behavior 






Average attendance rate (%) 

94.9 

94.4 

0.5 

0.05 

0.223 

Number of suspensions 

0.56 

0.59 

-0.03 

-0.01 

0.815 

Academic achievement 






Average core course marks (%) 

81.6 

81.7 

-0.1 

-0.01 

0.838 

Number of students (total = 910) 

442 

468 





(continued) 
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Table 4.3 (continued) 


SOURCE: MDRC calculations based on student records obtained from school districts. 

NOTES: The analyses reported in this table are based on the stable school records sample, which includes all 
students with course-failure data for the 2012-2013 and 2013-2014 school years. 

Estimated effects are regression-adjusted using ordinary least squares, controlling for the blocking of 
random assignment by school and grade level at random assignment, as well as the following baseline 
characteristics: race, gender, free or reduced-price lunch status, English as a second language, whether 
qualified for a gifted program, and special education status. 

The values in the column labeled “Case-Managed Group” are the observed means for students randomly 
assigned to the case managed group. The “Non-Case-Managed Group” values in the next column are the 
regression-adjusted means for students randomly assigned to the non-case-managed group, using the observed 
distribution of the case-managed group across random assignment blocks as the basis for the adjustment. 

Effect sizes are calculated by dividing the impact estimate by the standard deviation of the outcome 
measure for students in the stable school records sample who are in the non-case-managed group. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between the case-managed and non-case-managed groups. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a A student is considered to be chronically absent if he or she has an attendance rate of less than 90 percent. 


these transition grades than it does in other grades. After two years, effects on the study’s two 
main outcomes — chronic absenteeism and course failure — are not statistically significant for 
students who were first offered case management in sixth and ninth grade compared with non¬ 
case-managed students in those grades. However, the effects are significantly different from the 
effects on students first assigned to these services in nontransition grades. For example, 4.0 
percentage points fewer students who began case management in ninth grade were chronically 
absent in the second year compared with their non-case-managed peers. This -4.0 percentage 
point effect is not statistically significant. But among students who began case management in 
nontransition grades (that is, grades 10 and 11), 5.7 percent more of them were chronically 
absent in the second year compared with their non-case-managed peers. This 5.7 percentage 
point effect is also not statistically significant. However, the 9.7 percentage point difference 
between the -4.0 and 5.7 percentage point effects is statistically significant, suggesting that case 
management is making more of a desired difference for students who join caseloads in ninth 
grade than it is for students who join caseloads in later grades. 

In high schools, estimated effects on core credit accumulation follow a similar, but 
more striking pattern. After two years, students assigned to receive case management in ninth 
grade had earned significantly more credits than non-case-managed students (estimated effect = 
1.99 percentage points). Further, this effect is significantly larger than the effect on this outcome 
at the end of the first year, which suggests that the benefits of receiving case management 
accumulated over time. These results suggest that providing CIS case management to ninth- 
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grade students for multiple years may help them progress toward high school graduation. 
Conversely, for students who were assigned to receive case management beginning in tenth or 
eleventh grade, the estimated effect on core credit accumulation is statistically significant in the 
wrong direction (estimated effect = -3.40 percentage points), an effect which is significantly 
more negative than the effect at the end of the first year. Core credit accumulation is the only 
outcome for which there are statistically significant differences between the first- and second- 
year effects. 13 


Discussion 

The analyses conducted after the second follow-up year indicate that CIS case management had 
an effect on several nonacademic mediating outcomes, but that it did not have a statistically 
significant effect on the study’s main school outcomes — chronic absenteeism or the percentage 
of students failing at least one core course. 

Specifically, Level 2 case management had positive and statistically significant effects 
on students’ reports that they had caring adults at home, school, and outside of home or school, 
as well as on their reports of friendship quality, which suggests that case management may be 
helping case-managed students have more trusting comiections and networks of support with 
peers and adults compared with non-case-managed students. This finding comports with what 
many site coordinators describe as some of their primary responsibilities: setting up meetings 
with students and linking students to other resources and people inside and outside of school. 

However, Level 2 case management did not have a positive effect on more traditional 
school outcomes. In fact, the levels of chronic absenteeism and course failure increased at 
similar rates among both case-managed and non-case-managed students from the time they 
enrolled in the study to the end of the second follow-up year. In addition, case management did 


13 Also, the estimated effect on core credit accumulation at the end of the second year is significantly dif¬ 
ferent for students assigned to case management in ninth grade versus higher grades. See Appendix C for 
grade-level subgroup tables. 

The study team also examined whether the estimated effect of CIS case management varies among the 
schools in the study. The team found that the estimated effects on one of the two main outcomes — chronic 
absenteeism — do vary by a statistically significant amount among schools, but there is very little consistency 
across study years in which schools have the largest or smallest effects: The correlation between school-level 
effects in Year 1 and Year 2 is only 0.04. Nonetheless, because there is variation in effects among schools, the 
study team investigated whether there is an association between school-level effects on chronic absenteeism 
and various school-level features of the program (for example, caseload size, site coordinator education and 
experience, average hours and frequency of case management services provided to students, etc.) and school 
characteristics (size, need, staffing, etc.). These analyses did not generate any fruitful hypotheses as to why case 
management has a greater effect in some schools than others. None of the examined program features or school 
characteristics were consistently and reliably associated with effects at the school level. 
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not lead to statistically significant differences in attendance rates, credit earning, or course 
perfonnance between the case-managed and non-case-managed groups. Although CIS did have 
a statistically significant effect on the number of suspensions, this effect was in the wrong 
direction: Students in the case-managed group had more suspensions than students in the non¬ 
case-managed group. Overall, these results indicate that it cannot be concluded that CIS case 
management improved students’ attendance, course performance, or behavior. 
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Chapter 5 

Conclusion 


This chapter briefly reviews findings from this experimental study of case management and 
discusses some implications for practice, notably for Communities In Schools (CIS) but also for 
other organizations that organize or coordinate integrated student support services. The chapter 
then discusses the findings from the quasi-experimental study of the CIS whole-school model 
and reflects on the findings across both halves of this evaluation: whole-school and case 
management. 


What Effect Has CIS Case Management Had on Students? 

On average in the schools that participated in this evaluation, CIS case management succeeded 
in getting students to participate in more support activities and had an effect on students’ 
nonacademic mediating outcomes related to being connected to adults, maintaining relation¬ 
ships with more positive or supportive peers, and being connected to some services intended to 
support their perfomiance and progress in school. While these are encouraging findings that 
suggest case management is meaningfully affecting the experiences of students, within two 
years case management did not have positive effects on students’ school progress, achievement, 
attendance, or behavior. On average, outcomes in these domains worsened for the case- 
managed and non-case-managed students in the study sample. The only statistically significant 
difference between case-managed and non-case-managed students was in the average number 
of suspensions, where case-managed students averaged more suspensions than non-case- 
managed students. 


What Are the Implications for Practice Based on This Evaluation 
of Case Management? 

The evaluation findings suggest some areas where the CIS national office might consider 
providing more or different guidance and support to the affiliates and schools in its network. 
The random assignment study’s implementation findings may be particularly useful for inform¬ 
ing possible adjustments to CIS practice, and may reveal some areas other integrated student 
support organizations may also want to focus on. As discussed below, the CIS national office 
has already begun to implement changes based on the results of this evaluation. 

• This study found that students potentially at higher risk of dropping out re¬ 
ceived similar levels of service as those who may have been less at risk. The 
first report from this evaluation contained similar findings, and in response 
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CIS has already begun moving beyond the two-level model implemented by 
schools in this study to a three-tiered model intended to provide more inten¬ 
sive support to the students with the most need. Further, CIS has begun clas¬ 
sifying students based on their level of risk or need and is focusing on 
providing more differentiated support to students to meet their needs. 1 CIS 
should continue to pay attention to this issue, and collect data to monitor 
whether the three-tiered model delivers more appropriate services to stu¬ 
dents with greater needs. 

• About 80 percent of the case-managed student sample received case man¬ 
agement in the second study year. While keeping a large majority of students 
engaged for multiple years is a noteworthy accomplishment, it seems espe¬ 
cially challenging to provide continuous services to high-risk students, as the 
percentage of those students who received case management declined sub¬ 
stantially from the first to second year of the study. Other research has sug¬ 
gested that it may take at least two years for intensive case management to 
generate effects on student outcomes, 2 so CIS site coordinators and other 
practitioners might benefit from additional guidance about how to keep stu¬ 
dents engaged, particularly higher-risk students. 

• The data gathered for this evaluation also underscore the importance of being 
able to monitor student progress continually and adjust services as needed. 
During both years of the random assignment study, there was variation in the 
extent to which site coordinators monitored the outcomes of case-managed 
students, compared their progress with case plans, and revised case plans and 
service provision to keep services aligned with students’ needs. Other re¬ 
searchers studying case management programs have emphasized the im¬ 
portance of carefully assessing student needs and directing specific services 
to address individual students’ needs. 3 One of the recent changes the CIS na¬ 
tional office has made is to expect a minimum number of check-ins per stu¬ 
dent and to provide guidance to site coordinators about what should happen 
during the check-ins. CIS site coordinators may cdso benefit from guidance 
and support regarding how to monitor and adjust services, including the de- 


'This adjustment to the model occurred after the years of this study, and thus this study’s results do not 
capture the effect of this change. 

2 For example, Maynard, Kjellstrand, and Thompson (2014) reported that the Check & Connect dropout- 
prevention program had positive effects on attendance and behavior only when the program had been imple¬ 
mented for at least two years. 

3 Schirm, Stuart, and McKie (2006). 
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velopment of or training in systems intended to help them stay current about 
students ’ needs. 


• It can be inherently challenging for integrated student support organizations 
like CIS to monitor the quality of services provided by outside organizations 
and individuals. Case-managed students in this study did receive more ser¬ 
vices than non-case-managed students, but they did not have different school 
outcomes. It may be that the services they received did not address their most 
critical needs, or it may be that not all of the extra services they received 
were of high quality, and site coordinators could benefit from assistance to 
ensure their quality. Site coordinators rely regularly on others to provide di¬ 
rect services to students, and given their range of responsibilities and the pri¬ 
ority of their work with students, they may not be able to monitor the quality 
of those outside services. They may need additional support from their affili¬ 
ates to do so. This support could consist of more targeted training for site co¬ 
ordinators, or it could consist of a designated person at the affiliate level who 
monitors the quality of schools’ outside services. CIS could help school-level 
and affiliate staff members make decisions about which partners and prac¬ 
tices to bring into schools and about which in-school services to draw upon 
by giving them ways to assess the existing evidence about the effectiveness of 
these partners or practices , and by identifying evidence-based practices or 
organizations that might be a fit in their schools. The CIS national office has 
already begun moving in this direction by developing tools for the network 
regarding partnership engagement, with the aim of selecting high-quality, 
evidence-based providers. 

• Subgroup analyses for this study suggest that CIS case management may 
have the greatest potential to improve the outcomes of students who begin 
receiving it when they first enter middle or high school (that is, in sixth and 
ninth grades). CIS secondary school site coordinators typically bring new 
students onto their caseloads when they are in those transition grades, and 
continuing to do so may help maximize the impact of case management. 

Furthermore, other research about case management offers some additional considera¬ 
tions for case management practices: 

Case management priorities and practices should be responsive to the changing needs 
and experiences of students at different levels of schooling. One experimental evaluation of 
CASASTART, a program that targets 11- to 13-year-olds at risk for substance abuse, delinquent 
behavior, and negative school outcomes, found several statistically significant positive effects 
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based on two years of case management: reductions in drug use, drug sales, and violent crimes; 
improved peer support; and less association with delinquent peers. 4 An evaluation of the effect 
of the Quantum Opportunity Program (QOP) on students who were entering ninth grade when 
they joined found no effects on a variety of outcomes such as the delinquent behaviors studied 
in the CASASTART evaluations and no effects on earning a high school diploma or the 
equivalent, grades, test scores, course credits earned, or postsecondary education and employ¬ 
ment. The QOP evaluators note that students entering high school were already involved in 
risky behaviors that may have required different fomis of support than the program was 
prepared to provide. 5 Case management for middle school students may help to prevent behav¬ 
iors that those students have not yet have begun, while case management for high school 
students may require different approaches to address students’ existing behaviors. 6 Although 
this experimental evaluation of CIS case management did not find differences between the 
effects in middle schools and high schools, other research provides a reminder that case man¬ 
agement should recognize and respond to differences in students’ needs based on where they 
are in their developmental and educational trajectories. 

Identifying students in need is important, but difficult, and multiple sources of infor¬ 
mation should be used to identify at-risk students whenever possible. Students in need (and at 
risk of dropping out) are often identified based on available data indicating prior struggles, such 
as attendance issues or suspensions from school. However, research on dropout prevention 
suggests that individual risk factors alone are inefficient predictors of whether students will drop 
out. 7 That is, while they may identify many potential dropouts, they also identify other students 
who will not drop out and still miss some students who eventually do drop out. Using multiple 
risk factors, rather than individual ones, identifies potential dropouts more efficiently, but it is 
still an imperfect method. 8 


4 This experimental evaluation by Harrell, Cavanaugh, and Sridharan (1999) randomized young people 
within specific neighborhoods to participate in the program or not. The researchers also conducted a quasi- 
experimental evaluation that compared outcomes in these neighborhoods with similar high-risk neighborhoods. 
It did not find impacts. CASASTART was formerly known as Children at Risk, and is referred to by that name 
in the evaluation report. 

5 Shirm, Stuart, and McKie (2006). The researchers also note that in practice the program did not meet its 
target of 750 service hours per year, and schools had difficulty implementing the education/tutoring component 
of the model. 

furthermore, Dynarski et al. (1998) suggest that dropout-prevention interventions (including those that 
use case management) have a greater chance of benefiting middle school students than high school students. 
Balfanz, Herzog, and Mac Iver (2007) find that indicators of potential dropout are better predictors of later 
outcomes for students before or early in middle school than they are in high school, suggesting that the existing 
methods of identifying students in need of support may be more accurate if applied early. 

7 Tyler and Lofstrom (2009). 

8 Dynarski and Gleason (1998); Balfanz, Herzog, and Mac Iver (2007). 
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To improve these risk or needs assessments, researchers have suggested accounting for 
transitory events, psychological factors, and the persistence of risk characteristics over time, all 
of which can be difficult to assess . 9 If it were possible to identify or develop standardized 
assessments that measure social and emotional characteristics such as resiliency and conscien¬ 
tiousness, then those assessments could be used to more accurately identify students who need 
case management, and to decide which services might benefit them. CIS site coordinators often 
consider students for their caseloads based on referrals by other adults in the school. These 
referrals often take into account issues like transitory events in students’ lives. This study also 
found that some site coordinators identify students who might be in need of case management 
using school data about attendance, course performance, or disciplinary problems. The research 
cited here suggests that site coordinators may have the most success at identifying students in 
need when they draw on multiple sources and types of infomiation. 


Case Management Is One Component of the CIS Model. Does the 
Whole-School Model Make a Difference for Students? 

Although this study focused on the implementation and effects of case management 
alone, the larger evaluation included a separate study, as indicated at the beginning of the report, 
intended to assess the effect of the complete CIS whole-school model. Findings in this arm of 
the evaluation — which used a quasi-experimental, comparative interrupted time series design 
and included elementary, middle, and high schools — were mixed, but more promising . 10 
Results from this study include the following: 

• After three years of CIS implementation, on-time graduation rates and drop¬ 
out rates improved by statistically significant amounts in CIS high schools, 
relative to what would have been expected given their baseline trends. How¬ 
ever, it is not clear whether these improvements were caused by the CIS 
model. On the one hand, graduation and dropout rates improved more in the 
CIS high schools than in comparison high schools, which suggests that the 
CIS model may have improved schools’ graduation and dropout rates more 
than they would have improved otherwise. On the other hand, the compari¬ 
son high schools and CIS high schools had different graduation and dropout 
rates before the CIS model was implemented, so the comparison schools may 
not provide a credible reference point. For this reason, it is not possible to de¬ 
termine whether the CIS model is more effective than the dropout-prevention 
strategies used by the comparison schools, though the study’s findings do 


9 Dynarski and Gleason (1998). 

10 Somers and Haider (2017). 
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suggest that the CIS model may be at least as effective as those other ap¬ 
proaches. 

• State test scores did not improve in the CIS middle schools relative to what 
would have been expected given their baseline trends. During the same time 
period, state test scores did improve by a statistically significant amount in 
the comparison schools. CIS schools’ test scores appear to be lower than they 
would have been otherwise. There were no effects on state test scores in high 
schools or elementary schools. 

• Attendance rates improved in CIS elementary schools by a statistically sig¬ 
nificant amount relative to what would have been predicted given their base¬ 
line trend, and the improvement in attendance rates for these CIS schools was 
larger than the improvement in comparison schools by a statistically signifi¬ 
cant amount. The CIS model appears to have improved CIS elementary 
schools’ attendance rates more than they would have improved otherwise, an 
increase equivalent to about an additional day of school on average. There 
were no effects on attendance in high schools or middle schools. 

Because the findings from the whole-school study are based on a small number of pur¬ 
posefully selected schools that started implementing the model a decade ago, they may not 
represent the effect of the CIS model nationally as it exists today. Therefore, the results from 
this study should be considered alongside the results of other evaluations of the CIS model, two 
of which are worth noting. The first is a national quasi-experimental study of the CIS model 
conducted by ICF International. 11 The second is an (as yet) unpublished school-level random 
assignment evaluation of the CIS model in Chicago K-8 schools. 12 

A notable finding from this broader body of research is that in all three studies of the 
CIS model conducted thus far, there have been positive effects on the attendance rates of 
younger students. With respect to state test scores, the effect of the CIS model appears to 
depend on the local context: MDRC’s quasi-experimental study finds negative effects at the 
middle school level, whereas the Chicago study finds positive effects. This difference suggests 
that in some settings, the CIS model can have positive effects on students’ test scores and be 
more effective than the other strategies available. 

The body of findings in high schools is more difficult to interpret. The ICF study finds 
an improvement of 1.7 percentage points in ninth-grade students’ probability of graduating from 


11 ICF International (2008). 
12 Figlio (2015). 
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high school, but these effects are not statistically significant. In MDRC’s quasi-experimental 
study, graduation rates improved for the CIS schools after they launched the model, but it is 
unclear to what extent graduation rates would have improved had these schools not implement¬ 
ed the CIS model. 


What Has Been Learned from the Two Studies in This 
Evaluation? 

What sense can be made of the findings from the study of case management together with the 
findings from the study of the whole-school model? While the whole-school study showed that 
schools implementing the CIS model experienced improved graduation and dropout rates 
relative to what would have been predicted given their baseline trends, the study of case 
management found that after two years case management does not improve the student out¬ 
comes thought to predict graduation, though it does get students into more support activities and 
improves their nonacademic outcomes. It is important to note that the two studies included 
different schools and covered different time periods: The whole-school study spanned three 
years and the case management study covered only two. Nonetheless, given that all the study 
schools were implementing the full CIS model, do the two studies together provide insights into 
why the study of the full model finds increasing graduation rates in CIS high schools, while the 
case management study finds no effects on attendance, behavior, and course perfonnance 
measures that often predict graduation? 

There are several possible explanations for this combination of findings: 

• It may be that having a CIS site coordinator who works closely with a group 
of case-managed students allows other support staff members, such as guid¬ 
ance counselors and social workers, to work more with the non-case- 
managed students than they would have otherwise. If that were true, the 
school as a whole might improve because of CIS’s presence, even though the 
students receiving case management did not improve more than students 
randomly assigned to the non-case-managed group. The non-case-managed 
students would then have benefited indirectly from the presence of a CIS site 
coordinator practicing case management. 

• A second possibility is that the Level 1 services CIS provides, which are ac¬ 
cessible to the majority of students in a school, may change school-wide out¬ 
comes more than case management can, with its focus on a small minority of 
targeted students. This notion is supported by school leaders’ reports that CIS 
is an important part of their schools and that CIS is an essential provider of 
support services. 
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• Finally, while previous research indicates that attendance, behavior, and 
course performance are correlated with graduation, it may be that this study 
was not able to track case-managed students for long enough to see effects on 
those outcomes, or that those outcomes might not be the only factors that 
predict graduation. This study was only able to follow students for two years. 

It is possible that over a longer time, the positive effects of case management 
on students’ nonacademic outcomes would translate into positive effects on 
their more traditional school outcomes. Alternatively, it may be that those 
nonacademic improvements could end up making a difference in keeping 
students in school, even though they never affect the traditional school out¬ 
comes tracked by this study. 

It remains uncertain, however, whether CIS’s model makes a bigger difference than alternative 
approaches to school improvement. A random assignment study of the whole-school model 
may be the next step that would provide the most information about the model’s effect relative 
to that of other programs and strategies. If an evaluation included a cost study on the implemen¬ 
tation of the CIS model, along with cost data on the strategies and interventions used by the 
control schools, it could also determine the CIS model’s relative cost-effectiveness, which 
would ultimately provide the most useful decision-making infonnation for school districts. 

As CIS and other integrated student support organizations continue to work toward ad¬ 
dressing students’ needs, they can leam from both completed and ongoing research to refine 
their models. While graduation rates have risen in the last decade, it remains the case that far too 
many students drop out of school — roughly a million every year. 13 Many communities have 
support services available to address these students’ needs, but the services are often offered by 
many different organizations, which makes it difficult to coordinate them and difficult for 
students to make use of them. This evaluation suggests that whole-school models of integrated 
student support services do offer the promise of positive effects. However, in the actual imple¬ 
mentation of tiered-support models, it appears to be important to pay close attention to how that 
tiered support might improve conditions for students above and beyond the kinds of support 
already available in a school. In addition to implementing the changes discussed above, CIS has 
been making efforts to strengthen its model by identifying affiliates who will focus on expand¬ 
ing their growth and on trying out innovative strategies to increase their effect on the students 
and schools they serve. As CIS and other similar organizations continue to refine their models, 
they should pay particular attention to reaching the students who are most in need of support 
and to connecting students with high-quality services. In addition, they may want to target 
schools for CIS implementation that do not already provide a broad range of services for 


13 National Center for Education Statistics (2015). 
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students, or where service provision is particularly fragmented, as these may be the schools 
where they would have the most to offer. 
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Appendix A 


The Statistical Model and Statistical Power 




This appendix discusses various technical issues related to the estimation of program effects. 
The first section provides the statistical model used to estimate the effect of Communities In 
Schools (CIS) case management on student outcomes. The second section discusses the 
minimum detectable effect size for the main effect findings in the study. The final section 
presents effect estimates that are not adjusted for students’ baseline characteristics. 


Statistical Model for Estimating Effects 

The impact of CIS case management on student outcomes is estimated by fitting the following 
regression model to the relevant student sample (the stable school records sample or the stable 
student survey sample): 

Yi = m + 1*4 B ki + Is S s X si +Y, s 0) s M si + a h (1) 

where: 

Yj = the outcome of interest for student i 

Tj = one if student i was assigned to the case-managed group and zero otherwise 

= a set of K random assignment block indicators, equal to one if student i is in ran¬ 
dom assignment block k and zero otherwise 

X si = a set of S baseline characteristics for student i 

Mi = a set of S missing indicators for each of the student characteristics, coded one if 
missing and zero otherwise 

s, = an error tenn for student i. 

Therefore: 

P = the estimated effect of case management on outcome Y. 

The block indicators are included in the model to capture a central feature of the re¬ 
search design in which random assignment was conducted separately for each school. 1 Control¬ 
ling for random assignment blocks in the model also accounts for the clustering of student 


'in one school, random assignment was also conducted by grade level and gender because a specific num¬ 
ber of boys and girls in each grade had to be served. In total, there are 30 random assignment blocks in the full 
study sample, the stable student survey sample, and the stable school records sample. 
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outcomes by school, because it explains all of the between-school variation in student out¬ 
comes. 2 


Controlling for students’ baseline characteristics is not necessary for obtaining unbiased 
estimates of effects, because random assignment should ensure that the program and control 
group have similar observed and unobserved characteristics when they enrolled in the study. 3 
However, controlling for student characteristics can increase the precision of the effect esti¬ 
mates, because these characteristics explain part of the within-block variation in the outcome 
measure. Controlling for student characteristics can also be used as a “safeguard” to ensure that 
the program and control group are comparable in all characteristics. 4 (As a point of reference, 
unadjusted effect estimates are presented later in this appendix.) 

The statistical significance of effect estimates (and other estimates) in this report is as¬ 
sessed using a two-tailed t-test. In this report, statistical significance is based on a significance 
level of 10 percent. 

Finally, it is important to note that the estimated effects presented in this report are “in- 
tent-to-treaf ’ estimates of the effect of two years of CIS case management. Some students 
assigned to case management did not receive the intended two years of services (in the stable 
student survey sample, 10 percent of students in the case-managed group received one year of 
services, and 9 percent did not receive any services). Thus, the findings in this report represent 
the estimated effect of offering two years of case management to students rather than the effect 
of receiving two years of case management. Because students’ participation in educational 
interventions is typically voluntary, intent-to-treat estimates of the effect of offering a program 
or service are relevant to policy. 


2 The random assignment ratio differs across blocks (minimum = 0.27, maximum = 0.67, median = 0.51 in 
the full study sample). These differences in the random assignment ratio must be accounted for to obtain an 
unbiased estimate of effects. There are several ways to account for variation in the random assignment ratio. 
The two most common are to (1) “block-mean” center the covariates on the right-hand side of the model or (2) 
include block fixed effects in the model. Raudenbush (2009) shows that these two methods produce the same 
effect estimate. This model is based on the latter approach. 

3 The following covariates are included in the statistical model: whether the student has English as a second 
language, whether a student is qualified for a gifted program, a student’s special education status, whether a 
student is eligible for free or reduced-price lunch, the student’s race/ethnicity and gender, and a baseline 
measure of the outcome variable. These co variates were chosen because they are strong predictors of academic 
achievement; the decision about which covariates to include in the model was made before starting the impact 
analysis. 

4 In particular, when differences between the program and control group are between 0.05 and 0.25 stand¬ 
ard deviations (as they are in this study; see Appendix B), the What Works Clearinghouse recommends that 
these characteristics be included as covariates in the impact model. See What Works Clearinghouse (2014). 
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Minimum Detectable Effect Sizes 


This section examines how large the effect of CIS case management would have to be 
for the evaluation to be able to detect it. A common way to convey a study’s statistical power is 
through the minimum detectable effect (MDE) or the minimum detectable effect size (MDES). 
Formally, the MDE is the smallest true program effect that can be detected with a reasonable 
degree of power (in this case, 80 percent) for a given level of statistical significance (in this 
case, 10 percent for a two-tailed test). The MDES is the MDE scaled as an effect size — in other 
words, it is the MDE divided by the standard deviation of the outcome of interest. Effect sizes 
are used widely for measuring the effects of educational programs and are defined in terms of 
the underlying population’s standard deviation of student achievement. For example, an MDES 
of 0.20 indicates that an effect estimator can reliably detect a program-induced increase in 
student achievement that is equal to or greater than 0.20 standard deviations of the existing 
student distribution. 

The MDE and MDES for a study are a function of the standard error of the estimated 
program effect: 5 

MDE = M N _ B _ x *s.e.(fi) (2a) 


MDES 

where: 

s.e.(P) 


N 

B 

X 


= M 


N-B-X 


* 


s-e.j/3) 

(7 


(2b) 


= the standard error of the effect estimate 

= the standard deviation that is used to calculate effect sizes (for example, in 
this study, it is the standard deviation of the non-case-managed group) 

= the number of students in the sample 

= the number of random assignment blocks in the impact analysis 

= the number of student baseline characteristics and missing-data indicator 
variables included as covariates in the impact model (see previous section) 


5 They are so because the standard error of the effect estimate is what determines whether the effect esti¬ 
mate is statistically significant. 
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Mn-b-x = the “degrees-of-freedom” multiplier, which is calculated to be 2.5 in this 
study, assuming a two-tailed test with a statistical power level of 0.80 and 
a statistical significance level of 0.10. 

Appendix Table A.l presents the MDES after two years of services for the outcomes in 
this report measured using school records, including the study’s two main outcomes (chronic 
absenteeism and course failure). As shown in this table, in the second follow-up year, the study 
is able to detect an effect of 4.9 percentage points on chronic absenteeism (an effect size of 
0.12) and an effect of 5.5 percentage points on the percentage of students who fail at least one 
course (an effect size of 0.12). 


Unadjusted Effect Estimates 

As explained earlier, the statistical model used to estimate effects controls for several measures 
of students’ baseline characteristics and prior achievement (see Equation 1). Although it is not 
strictly necessary to control for these baseline characteristics when using a random assignment 
design, the main impact analysis does so in order to improve the precision of the impact 
estimates. Controlling for students’ baseline characteristics should not appreciably affect the 
estimated effect — but it should reduce its standard error. 

To confirm that this expectation is true, Appendix Table A.2 compares estimated ef¬ 
fects from Chapter 4 (which are adjusted for student baseline characteristics) with effect 
estimates that are adjusted for blocking only (not adjusted for student characteristics). The table 
also shows the standard error of these effect estimates. As expected, controlling for student 
characteristics does not affect the magnitude of the estimated effects, but it does decrease their 
standard error. The extent to which controlling for student characteristics reduces the standard 
error varies among outcomes — from a reduction of 0.1 percent in the standard error for the 
impact on chronic absenteeism to a reduction of 3.4 percent in the standard error for the effect 
on core credit accumulation. 



Appendix Table A.l 


Minimum Detectable Effects and Effect Sizes 
for Effects on Academic and Behavioral Outcomes 


Outcome 


Number of 

Students MDE MDES 


Main outcomes 
Chronic absenteeism 3 (%) 

Failed at least 1 core course (%) 

School progress” 

Core credit accumulation for graduation (%) 
Behavior 

Average attendance rate (%) 

Number of suspensions 

Academic achievement 
Average core course marks (%) 


1,493 

4.91 

0.12 

1,501 

5.54 

0.12 

413 

2.87 

0.19 

1,493 

1.08 

0.00 

1,501 

0.42 

0.00 

1,501 

0.90 

0.00 


SOURCE: MDRC calculations based on student records obtained from school districts. 


NOTES: The analyses reported in this table are based on the stable school records sample, which 
includes all students with course-failure data for the 2012-2013 and 2013-2014 school years. 

The minimum detectable effect (MDE) and minimum detectable effect size (MDES) in this 
table are calculated based on the standard error of the impact estimate (adjusted for random 
assignment blocks and student baseline characteristics) and the number of students in the school 
records analysis sample. A statistical significance level of 10 percent is assumed. The MDES is 
calculated by dividing the MDE by the standard deviation of the outcome measure for students in 
the school records analysis sample who are in the non-case-managed group. 

a A student is considered chronically absent if he or she has an attendance rate below 90 
percent. 

b This outcome is for high school students only; the state-level graduation requirements for the 
four core subjects are used in this calculation. A student can be expected to accumulate around 25 
percent of core credits in each year of high school. 
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Appendix Table A.2 


Estimated Effects on Students’ Academic and Behavorial Outcomes, 
Adjusted and Unadjusted for Students’ Baseline Characteristics 



Adjusted for Blocking and 

Full Set of Student Characteristics 3 

Adjusted for Blocking Only 

Outcome 

Estimated 

Effect 

(S.E.) 

Effect 

Size 

P-Value for 

Estimated 

Effect 

Estimated 

Effect 

(S-E.) 

Effect 

Size 

P-Value for 

Estimated 

Effect 

Main outcomes 

Chronic absenteeism 11 (%) 

-0.07 

(1.974) 

0.00 

0.973 

-0.25 

(1.975) 

-0.01 

0.897 

Failed at least 1 core course (%) 

0.62 

(2.227) 

0.01 

0.780 

0.75 

(2.244) 

0.02 

0.740 

School progress' 1 

Core credit accumulation for graduation (%) 

-0.84 

(1.150) 

-0.06 

0.464 

-0.88 

(1.191) 

-0.06 

0.459 

Behavior 

Average attendance rate (%) 

0.35 

(0.432) 

0.04 

0.422 

0.29 

(0.445) 

0.03 

0.510 

Number of suspensions 

0.33 * 
(0.167) 

0.11 

0.051 

0.33 ** 
(0.167) 

0.11 

0.049 

Academic achievement 

Average core course marks (%) 

-0.25 

(0.360) 

-0.03 

0.483 

-0.37 

(0.365) 

-0.05 

0.310 

Number of students 

1,501 



1,501 




(continued) 





Appendix Table A.2 (continued) 


SOURCE: MDRC calculations based on student records obtained from school districts. 

NOTES: The analyses reported in this table are based on the stable school records sample, 
which includes all students with course-failure data for the 2012-2013 and 2013-2014 school 
years. 

All estimated effects are regression-adjusted using ordinary least squares, controlling for 
random assignment blocks by school and grade level. 

“S.E.” indicates standard error, given in parentheses. 

Effect sizes are calculated by dividing the impact estimate by the standard deviation of the 
outcome measure for students in the school records analysis sample who are in the non-case- 
managed group. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between case-managed and non-case-managed 
groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 
percent. 

The sample size reported in the table is for the stable school records sample. However, the 
sample size varies across outcomes due to missing data. The percentage of missing data is 0.5 
percent for attendance rates and absenteeism, 11 percent for credit accumulation, and 0 percent 
for other outcomes. 

a Estimated effects are adjusted for blocking and the following variables: race, gender, free or 
reduced-price lunch status, English as a second language, whether qualified for a gifted 
program, special education status, and a baseline measure of the outcome variable. 

b A student is considered to be chronically absent if he or she has an attendance rate below 90 
percent. 

c This outcome is for high school students only; the state-level graduation requirements for 
the four core subjects are used in this calculation. A student can be expected to accumulate 
around 25 percent of core credits in each year of high school. 
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Appendix B 


Samples and Response Rates 




This appendix provides additional infonnation about the samples of students in the study and 
response rates. The three exhibits for this appendix show how the different student samples 
were formed, provide baseline characteristics for the full study sample, and give the survey 
response and school record response rates for case-managed and non-case-managed students. 

Appendix Figure B. 1 shows how the original pool of students recruited for participation 
in the evaluation was reduced to the samples of students whose data were analyzed in this 
report. The eligibility pool of 2,578 represents all recruited students whose parents consented to 
their participation in Communities In Schools (CIS) case management. 1 To provide all students 
with the same opportunity to receive CIS case management, MDRC implemented a random 
assignment process that assigned students to one of three groups: a case-managed group to fill 
open caseload slots in participating schools (1,179 students), a non-case-managed control group 
(1,118 students), and a “wait-list” group (281 students who could fill caseload slots that might 
open during the course of the school year but who would not participate in the research activi¬ 
ties and would not be included in any analyses). In the second year of the study, 533 of the 
students assigned to the first two groups were removed from the study sample for one of the 
following reasons: (1) because they consented to participate only in CIS case management and 
not the evaluation, (2) because they left their participating schools after submitting consent 
fomis but before random assignment had been conducted, or (3) because they graduated or their 
districts chose not to participate in the second year of the study. 2 This left a full study sample of 
898 case-managed students and 866 non-case-managed students, or a total of 1,764 students. 

However, follow-up data on student outcomes were not available for all students in the 
full study sample. Of the 1,764 students in the full study sample, 348 did not respond to the 
surveys administered at the end of the first year (2012-2013) or the second year (2013-2014) of 
the study, or both. Similarly, the study team was unable to obtain records data for 263 of the 
study students. In this report, therefore, effects on outcomes that are measured using survey 
responses are based on the stable student survey sample, which is defined as students who 
responded to both follow-up surveys (1,416 students). Effects on outcomes measured using 
student records are based on the stable school records sample, which is defined as students for 
whom school records are available in both follow-up years (1,501 students). As mentioned in 
Chapter 3, these two samples overlap a great deal: 89 percent of students in the stable school 
records sample are in the stable student survey sample, and 95 percent of students in the stable 
student survey sample are in the stable school records sample. There is also overlap between the 


'Students who were 18 years of age or older — adults — were able to sign consent forms for themselves 
and did not need parental consent to participate. 

‘Although these students who left their schools did so before random assignment, the research team was 
not informed about their exits until after random assignment occurred. 
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Appendix Figure B.l 

Creation of the Student Samples in the Second Year 
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full study sample and both stable samples (the stable school records sample and the stable 
student survey sample): 80 percent of the full study sample is represented in the stable student 
survey sample and 85 percent of the full study sample is represented in the stable school records 
sample. 


Appendix Table B.l presents the baseline characteristics of the 1,764 students in the full 
study sample. As Chapter 3 shows for the stable student survey sample and the stable school 
records sample, the case-managed and non-case-managed students in the full study sample are 
highly similar, as indicated by the p-value of 0.979 for the overall test of difference that includes 
all measured characteristics. Also, the case-managed and non-case-managed groups’ values on 
these baseline measures are very similar to the values presented separately for the stable student 
survey sample and the stable school records sample in Tables 3.1 and 3.2. 

Appendix Table B.2 compares the survey response rates of the case-managed and non¬ 
case-managed students and gives the percentage of students in each of those two groups for 
whom school records data were obtained, in the first and second year of the study. The table 
also shows the percentages of students with data in both years (that is, the percentage of 
students who are included in the stable student survey and stable school records samples). 
Overall, the response rates for the two data sources (school records and survey) were high in the 
first year — over 90 percent. In the second year of the study, the response rates for the survey 
and school records were lower, between 80 and 87 percent. The differences in response rates 
between the case-managed and non-case-managed students are small in both years, ranging 
from 1.1 to 3.6 percentage points depending on the data source and study year. These differ¬ 
ences qualify as “low differential attrition” based on the current standards of the What Works 
Clearinghouse. Those standards indicate that for overall attrition of 20 percent, a difference 
between experimental groups of less than 5.4 percentage points is considered to be “low 
differential attrition.” 3 


3 What Works Clearinghouse (2014). 
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Appendix Table B.l 


Baseline Characteristics of Students in the 
Full Study Sample 


Characteristic 

Case- Non-Case- 

Managed Managed Estimated Effect 
Group Group Difference Size 

P-Value for 
Estimated 
Difference 

Race/ethnicity (%) 





0.175 

Hispanic 

60.7 

61.1 

-0.4 

0.0 


Black, non-Hispanic 

33.8 

30.5 

3.3 

0.1 


White, non-Hispanic 

3.0 

4.7 

-1.7 

-0.1 


Asian 

1.5 

2.2 

-0.8 

-0.1 


Other 

0.8 

1.5 

-0.7 

-0.1 


Male (%) 

44.2 

43.1 

1.1 

0.0 

0.642 

Eligible for free or reduced-price lunch (%) 

46.7 

44.7 

2.0 

0.0 

0.177 

English as a second language (%) 

13.5 

12.8 

0.7 

0.0 

0.670 

Qualified for a gifted program (%) 

2.4 

2.7 

-0.3 

0.0 

0.725 

Chronically absent (%) 

9.7 

10.3 

-0.6 

0.0 

0.715 

Average attendance rate (%) 

95.5 

95.5 

0.1 

0.0 

0.765 

Failed at least 1 core course (%) 

24.0 

23.5 

0.5 

0.0 

0.794 

Average core course marks (%) 

80.4 

80.5 

-0.1 

0.0 

0.694 

Household makeup 3 (%) 






Lives with 2 or more parents/guardians 

54.5 

53.5 

1.0 

0.0 

0.680 

Lives with 1 parent/guardian 

38.7 

39.1 

-0.3 

0.0 

0.888 

Lives with 1 or more grandparent(s) 

7.9 

9.1 

-1.2 

0.0 

0.389 

Lives with his/her own child 

2.0 

1.9 

0.1 

0.0 

0.867 

Language predominantly spoken at home* 3 (%) 






English 

75.0 

74.1 

1.0 

0.0 

0.650 

Not English 

25.0 

25.9 

-1.0 

0.0 

0.650 

Parent educational attainment (%) 






Father 





0.643 

Not a high school graduate 

15.4 

19.2 

-3.8 

-0.1 


High school graduate or equivalent 

24.2 

20.9 

3.3 

0.1 


College graduate or higher 

12.6 

13.5 

-1.0 

0.0 


Don’t know 

47.9 

46.4 

1.5 

0.0 


Mother 





0.709 

Not a high school graduate 

16.9 

20.6 

-3.8 

-0.1 


High school graduate or equivalent 

26.5 

26.2 

0.3 

0.0 


College graduate or higher 

23.5 

22.8 

0.7 

0.0 


Don’t know 

33.1 

30.3 

2.7 

0.1 


(continued) 
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Appendix Table B.l (continued) 



Case- 

Non-Case- 



P-Value for 


Managed 

Managed 

Estimated 

Effect 

Estimated 

Characteristic 

Group 

Group Difference 

Size 

Difference 

Did any siblings leave high school before 
graduation? 0 (%) 






None left high school 

64.7 

68.6 

-3.9 

-0.1 

0.171 

At least 1 left high school 

35.3 

31.4 

3.9 

0.1 

0.171 

Student engagement with school (l-4) d 

2.85 

2.90 

-0.06 

-0.1 

0.119 

How far would you like to go in school? (%) 





0.989 

Some high school 

0.8 

1.1 

-0.3 

0.0 


Finish high school 

12.4 

9.6 

2.8 

0.1 


Some college or trade/technical school 

5.4 

5.2 

0.2 

0.0 


Finish college or trade/technical school 

42.2 

43.7 

-1.5 

0.0 


Graduate school after college 

31.6 

31.9 

-0.4 

0.0 


Don’t know 

7.7 

8.5 

-0.8 

0.0 


How far do you think you will actually go in 
school? (%) 





0.984 

Some high school 

1.9 

1.1 

0.8 

0.1 


Finish high school 

14.1 

14.3 

-0.2 

0.0 


Some college or trade/technical school 

11.1 

11.1 

0.0 

0.0 


Finish college or trade/technical school 

35.4 

38.1 

-2.7 

-0.1 


Graduate school after college 

25.9 

25.1 

0.8 

0.0 


Don’t know 

11.6 

10.3 

1.3 

0.0 


Caring adult at home (l-4) e 

3.38 

3.40 

-0.02 

0.0 

0.446 

Caring adult at school (1-4) 

Caring adult outside of home 

3.30 

3.27 

0.03 

0.0 

0.338 

or school (l-4) g 

3.44 

3.48 

-0.04 

-0.1 

0.180 

Joint test of difference between groups 11 

(X 2 = 61.4) 




0.979 

Number of students (total = 1,764) 

898 

866 





(continued) 
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Appendix Table B.l (continued) 

SOURCES: MDRC calculations based on school records and the fall 2012 baseline student survey. 

NOTES: The analyses reported in this table are based on the full study sample, which includes all 
students the study team expected to receive in the second year of the study (that is, those who did not 
graduate and who were not part of the affiliate that dropped out of the study). Due to small numbers, 
percentages for the Native American demographic group are not included in the table. 

Questions with the same wording as those in the baseline student survey appear in the follow-up 
student survey in Appendix D. 

The estimated differences between the case-managed group and the non-case-managed group are 
regression-adjusted using ordinary least squares, controlling for random assignment blocks by school. 
The values in the column labeled “Case-Managed Group” are the observed means for students 
randomly assigned to the case-managed group. The “Non-Case-Managed Group” values are the 
regression-adjusted means for students randomly assigned to the non-case-managed group, using the 
observed distribution of the case-managed group across random assignment blocks as the basis for the 
adjustment. 

Effect sizes are calculated by dividing the estimated difference by the standard deviation of the 
characteristic for students in the full study sample who are in the non-case-managed group. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between the case-managed and non-case-managed 
groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Due to missing values, the number of students included varies by characteristic. The sample size 
reported here is for the full study sample. The average percentage of missing data for any given 
characteristic is 11 percent and ranges from 0 percent to 38 percent. 

“These survey categories are not mutually exclusive. 

b Of those students who responded that English was not the predominant language at home, 92 
percent reported speaking Spanish at home. 

“Respondents without siblings old enough for high school are omitted. 

d Scale based on responses to survey questions 9a-9e, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach’s alpha = 0.83. 

“Scale based on responses to survey questions 17a-17g, ranging from 1 = "not at all true” to 4 = 
“very much true.” Cronbach’s alpha = 0.81. 

f Scale based on responses to survey questions 8a-8f, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach’s alpha = 0.89. 

g Scale based on responses to survey questions 15a-15f, ranging from 1 = “not at all true” to 4 = 
“very much true.” Cronbach’s alpha = 0.89. 

h A chi-square test was used to determine whether there is a systematic difference between the case- 
managed group and the non-case-managed group when they joined the study, based on the 
characteristics included in this table as well as indicators of missing data for all relevant student 
characteristics. 


100 



Appendix Table B.2 


Percentages of Full Study Sample Students Included in the Analysis 


Data Source for Outcomes (%) 

Case- 

Managed 

Group 

Non-Case- 

Managed 

Group 

Estimated 

Difference 

P- Value for 

Estimated 

Difference 

School records 

Year 1 

95.8 

96.9 

- 1.1 

0.215 

Year 2 

84.2 

87.1 

-2.9 * 

0.082 

Stable school records sample 3 

83.6 

86.4 

- 2.8 

0.102 

Survey 

Year 1 

92.4 

95.3 

_2 9 ** 

0.011 

Year 2 

79.5 

82.8 

-3.3 * 

0.080 

Stable student survey sample b 

78.1 

81.6 

-3.6 * 

0.061 

Number of students (total = 1,764) 

898 

866 




SOURCES: MDRC calculations based on the spring 2013 and 2014 follow-up student surveys 
and student records obtained from school districts for the 2012-2013 and 2013-2014 school 
years. 


NOTES: The analyses reported in this table are based on the full study sample. The analyses 
reported in this table are based on the full study sample, which includes all students the study 
team expected to receive in the second year of the study (that is, those who did not graduate 
and who were not part of the affdiate that dropped out of the study). 

The estimated differences between the case-managed group and the non-case-managed 
group are regression-adjusted using ordinary least squares, controlling for random assignment 
blocks by school. The values in the column labeled “Case-Managed Group” are the observed 
response rate for students randomly assigned to the case-managed group. The “Non-Case- 
Managed Group” values are the regression-adjusted response rates for students randomly 
assigned to the non-case-managed group, using the observed distribution of the case-managed 
group across random assignment blocks as the basis for the adjustment. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between case-managed and non-case- 
managed groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 
percent; * = 10 percent. 

a The stable school records sample includes students who had data on course failure in both 
Year 1 and Year 2 of the study. 

b The stable student survey sample includes students who responded to both the first (spring 
2013) and second (spring 2014) student surveys. 
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Appendix C 


Additional Results 




This appendix provides information about additional analyses conducted by the research team. 
The various exhibits in this appendix show the estimated effects of Communities In Schools 
(CIS) case management on different subgroups of students, as well as estimated effects in the 
first year of the study (after one year of case management). 

Appendix Table C.l shows the services received by case-managed students in the first 
year of the study (2012-2013). Around 91 percent of case-managed students received a service 
at least once during the school year. These students received services an average of nearly 17 
times during the year for an average of 16.9 total service hours. In other words, students 
receiving case management in the first year of the study received services at slightly lower rates 
and for slightly fewer hours than students in the second year of the study (19.9 service contacts 
and 18.4 service hours, as shown in Table 3.3). 

Appendix Table C.2 presents a supplementary analysis of the services received by stu¬ 
dents in the case-managed and the non-case-managed groups. It shows the number of times per 
year that students in each group participated in in-school support activities and in-school 
meetings with adults. (This table is the counterpart of Table 3.4 in Chapter 3, which shows the 
proportion of students in each group who received these services.) For all of the activities listed 
in Appendix Table C.2, case-managed students participated more frequently than non-case- 
managed students (whether the difference is statistically significant or not). For example, case- 
managed students met with mentors around four times per year, whereas students in the non¬ 
case-managed group met with mentors around twice per year (p-value for the difference = 
0 . 000 ). 


Appendix Table C.3 shows the services received by high- and moderate-risk students. 
For a number of the services, there are statistically significant differences in service receipt 
between case-managed and non-case-managed students in one or both of the two subgroups. 
However, in general the contrast in service receipt was not consistently or significantly greater 
for high-risk students than it was for moderate-risk students (or vice versa). 1 The table also 
shows that two subgroups received similar levels of services: High-risk students in the case- 
managed group did not necessarily receive more services than their moderate-risk peers. 

Appendix Tables C.4 and C.5 show the first-year results for the study’s nonacademic 
mediating outcomes and traditional school outcomes. The pattern of results is similar to that 
shown in Chapter 4. After one year of implementation, case-managed students reported more 


t he one exception is with respect to the number of meetings with a mentor. Among high-risk students, 
students in the case-managed group met with mentors 3.1 times more often than students in the non-case- 
managed group. Among moderate-risk students, the difference between the case-managed and non-case 
managed group is 1.3 times. The service contrast for the high-risk group (3.1 times) is greater than the service 
contrast for the moderate-risk subgroup (1.3 times) by a statistically significant amount. 
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caring relationships with adults at home, at school, and outside of home and school than did 
non-case-managed students. Similarly, case-managed students had more positive views of 
education than non-case-managed students after one year. These results are statistically signifi¬ 
cant. For the main outcomes (as shown in Appendix Table C.5), there are no statistically 
significant differences between case-managed and non-case-managed students in chronic 
absenteeism or in the percentage of students who failed at least one core class. Finally, there are 
also no statistically significant differences between the two groups in any of the other outcomes 
of school progress, student behavior, or academic achievement. 

Appendix Tables C.6 and C.7 examine whether the grade level at which students are as¬ 
signed to case management has an effect on this study’s main outcomes — chronic absenteeism 
and core course failure — or on school progress, behavior, attitudes, or academic achievement. 
Table C.6 presents the two-year effects of CIS case management on students who were in ninth 
grade at the time of random assignment, as well as on their tenth- and eleventh-grade counter¬ 
parts. While case-managed students who entered ninth grade at the beginning of the study 
outperformed their peers in the non-case-managed group in chronic absenteeism, core credit 
accumulation (p-value for the estimated difference = 0.066), attendance, and course marks, they 
were also slightly more likely to be suspended (p-value for the estimated difference = 0.053) 
and fail a core course. In contrast, students who began receiving case management later in their 
high school careers were considerably more likely to fail at least one core course, accumulate 
fewer credits toward graduation, and achieve lower course marks after two years than students 
in the non-case-managed group (all significant at p = 0.05 or lower). Although not all effects 
were statistically significant, the ninth- and non-ninth-grade subgroups trend in the opposite 
directions for almost every outcome measure, suggesting that ninth-graders may benefit from 
CIS case management more than tenth- and eleventh-graders. 

Appendix Table C.7 shows the results of a parallel analysis for middle school students, 
examining the two-year effects of case management on students who began receiving services 
in sixth grade, another transition year, compared with those who began in seventh or eighth 
grade. While the results are not statistically significant for any outcome, the outcome levels for 
middle school students further support the notion that CIS has a greater effect when implement¬ 
ed at important junctures where students are most likely to fall offtrack. Students in sixth grade 
who received case management performed better than their peers in the non-case-managed 
group on all outcome measures, with the exception of number of suspensions. Conversely, 
seventh- and eighth-grade case-managed students were outperformed by their counterparts in 
the non-case-managed group on all applicable measures. 

Taken together, the results presented in Appendix Tables C.6 and C.7, while largely not 
statistically significant, suggest that CIS may be better able to influence students’ trajectories if 
it intervenes in a transition year — sixth or ninth grade. 
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Appendix Table C.l 

Services Received by Case-Managed Students in Year 1 


Service Received 

Percentage 

of Students 

Receiving Service 

Average Number 

of Times Service 

Was Received 

Average 

Total Hours of 

Service Received 

Overall 

Any service type 

91.3 

NA 

NA 

Across all service types 

NA 

16.5 

16.9 

By service tvne 

Academic 

83.6 

4.1 

4.9 

Behavioral 

53.5 

2.9 

3.8 

Social/life skills 

60.2 

5.0 

6.0 

Resources to meet basic needs 

61.8 

1.6 

2.5 

College/career preparation 

40.7 

0.9 

4.7 

Enrichment/motivation 

49.8 

1.0 

4.1 

Family-related 

51.4 

1.2 

0.4 

Attendance 

16.8 

0.7 

0.3 

Number of students 



701 


SOURCES: The Communities In Schools Data Management system (CISDM) and two local CIS 
service-provision databases (2013-2014). 

NOTES: The analyses above are for case-managed students only, and are based on the stable 
student survey sample, which includes all students who responded to the spring 2013 and 2014 
follow-up student surveys. Outliers and students with missing data are excluded from the analyses; 
for details, see Appendix E. 

The services offered are not mutually exclusive; a student could have received more than one 
type of service over his or her enrolled period. 

Calculations for the percentage of students receiving a given service are based on a consistent 
denominator of 701 case-managed students. However, information on all service types was not 
available in each data source. All data sources provided information on academic, behavioral, and 
social/life-skills services. 

Rounding may cause slight discrepancies in averages and percentages. 

NA = not applicable. 

The services described in this table were received over the entire time students were enrolled in 
CIS case management during the 2013-2014 school year. The averages presented for number of 
times and total hours of service include only those students who received the service in question. 

Of all case-managed group students (701), 8.7 percent did not receive case management. In 
addition, of all non-case-managed group students (715), 1.5 percent received case management 
but are not included in this table. 


107 






Appendix Table C.2 

Effects on the Times Per Year Students Received Services 
According to the Student Survey 


Survey Item 

Case- 

Managed 

Group 

Non-Case- 

Managed 

Group 

Estimated 

Difference 

P-Value for 

Estimated 

Difference 

In-school support activities 3 

Tutoring or homework help 

7.0 

6.0 

1.0 ** 

0.026 

Mentoring 

3.7 

1.8 

^ Q H 5 

0.000 

Community service/volunteering 

2.9 

2.5 

0.4 

0.251 

Exercise class or club f 

1.5 

2.2 

-0.7 ** 

0.016 

Positive behavior program 
such as drug-free/antibullying 

2.3 

2.0 

0.3 

0.337 

College planning activity 

5.0 

4.6 

0.4 

0.408 

Career planning activity 

3.4 

2.6 

0.8 ** 

0.021 

Job shadowing/internship f 

1.6 

1.2 

0.4 

0.123 

After-school program 

4.4 

4.6 

-0.3 

0.550 

Assistance such as school supplies, 
food, bus pass, clothing, or gifts 

2.4 

2.3 

0.2 

0.621 

Health check-up 

2.8 

2.6 

0.2 

0.474 

In-school meetings with adults 3 

Individual meeting about academics 

6.9 

5.6 

Y ^ *** 

0.005 

Individual meeting for support during a 
life-changing event 

2.6 

1.8 

0.7 ** 

0.025 

Individual meeting about personal 
goals and behavior 

4.9 

4.1 

0.7 

0.105 

Group meeting about academics 

4.9 

3.6 

^ “I s 

0.004 

Group meeting for support during 
a life-changing event 

2.0 

1.7 

0.3 

0.275 

Group meeting about personal 
goals and behavior 

3.5 

3.3 

0.2 

0.655 

Group meeting for social activities 

5.1 

2.7 

2 ^ i f : ^*1* 

0.000 


(continued) 
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Appendix Table C.2 (continued) 


Survey Item 

Case 

Managed 

Group 

Non-Case 

Managed 

Group 

Estimated 

Difference 

P-Value for 

Estimated 

Difference 

Met with an adult at school to set 
specific goals for the year (%) 

67.8 

49.9 

Y q g *** 

0.000 

Could earn rewards for improving grades, 
attendance, behavior, or for reaching 
or making progress toward goals (%) 

59.4 

53.6 

5.7 ** 

0.029 

An adult in school connected student to support 
programs or help outside of school (%) 

50.9 

39.8 

Y y y *** 

0.000 

Number of students' 3 (total = 1,416) 

701 

715 



SOURCE: MDRC calculations based on the spring 

2014 follow- 

-up student: 

survey. 



NOTES: The analyses reported in this table are based on the stable student survey sample, which 
includes all students who responded to the spring 2013 and spring 2014 follow-up student surveys. 

The estimated differences between the case-managed group and the non-case-managed group are 
regression-adjusted, controlling for the blocking of random assignment by school, as well as the 
following baseline characteristics: race, gender, free and reduced-price lunch, English as a second 
language, whether qualified for a gifted program, and special education status. 

The values in the column labeled “Case-Managed Group” are the observed means for students 
randomly assigned to the case-managed group. The “Non-Case-Managed Group” values are the 
regression-adjusted means for students randomly assigned to the non-case-managed group, using the 
observed mean covariate values for the case-managed group as the basis for the adjustment. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between the case-managed and non-case-managed 
groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The sample size reported in the table is for the stable student survey sample. However, the sample size 
varies across outcomes due to missing data. The percentage of missing data on any given outcome does 
not exceed 4 percent. 

A dagger (f) indicates that there is a statistically significant difference between the estimated effects 
in Year 1 and Year 2 at the 5 percent level. 

a The times-per-year measure was created by combining a measure of duration (“for how long did you 
do this activity?”) and a measure of frequency (“how often did you do this activity?”) for each of the 
above support activities in the spring 2014 follow-up student survey. The duration measure was 
converted from a four-point scale to an estimate of the number of weeks per school year a student 
participated in the activity: “1 never did this activity” = 0 weeks, “less than half of the school year” = 9 
weeks, “about half of the school year” =18 weeks, and “most or all of the school year” = 27 weeks. The 
frequency measure was converted from a four-point scale to an estimate of the number of times per week 
a student participated in the given activity: “1 never did this activity” = 0, “less than once a month” = 
0.125, “1-2 times a month” = 0.375, and “one or more times a week” = 1. 
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Appendix Table C.3 

Effects on Times Per Year Students Received Services 
According to the Student Survey, by Risk Category 



Case- 

Non-Case- 


P-Value for 


Managed 

Managed 

Estimated 

Estimated 

Survey Item 

Group 

Group 

Difference 

Difference 

In-school support activities (times per vear) a 

Received tutoring or homework help 





High-risk students 

7.8 

6.3 

1.5 ** 

0.045 

Moderate-risk students 

6.5 

5.8 

0.7 

0.211 

Met with a mentor f 





High-risk students 

5.0 

2.0 


0.000 

Moderate-risk students 

2.8 

1.6 

^ -fa -fa 

0.002 

Community service/volunteering 





High-risk students 

2.7 

2.6 

0.1 

0.787 

Moderate-risk students 

3.0 

2.5 

0.5 

0.216 

Exercise class or club 





High-risk students 

1.3 

1.6 

-0.3 

0.498 

Moderate-risk students 

1.6 

2.6 

- 1.0 ** 

0.017 

Positive behavior program such as drug-free/antibullying 




High-risk students 

2.3 

2.3 

0.0 

0.953 

Moderate-risk students 

2.3 

1.9 

0.5 

0.241 

College planning activity 





High-risk students 

5.0 

4.3 

0.7 

0.293 

Moderate-risk students 

4.9 

4.8 

0.1 

0.803 

Career-planning activity 





High-risk students 

3.3 

2.5 

0.8 

0.168 

Moderate-risk students 

3.4 

2.6 

0.8 * 

0.067 

Job shadowing/internship 





High-risk students 

2.1 

1.3 

09 ** 

0.046 

Moderate-risk students 

1.3 

1.2 

0.1 

0.857 

After-school program 





High-risk students 

4.9 

4.4 

0.5 

0.529 

Moderate-risk students 

4.0 

4.8 

- 0.8 

0.203 

Assistance such as school supplies, food, bus pass, clothing, or 

gifts 



High-risk students 

2.7 

2.5 

0.2 

0.738 

Moderate-risk students 

2.2 

2.1 

0.2 

0.718 

Health check-up 





High-risk students 

2.6 

2.6 

0.0 

0.958 

Moderate-risk students 

2.9 

2.6 

0.4 

0.360 

(continued) 
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Appendix Table C.3 (continued) 



Case- 

Non-Case- 


P-Value for 

Managed 

Managed 

Estimated 

Estimated 

Item 

Group 

Group 

Difference 

Difference 

In-school meetings with adults (times per vear) a 

Individual meeting about academics 





High-risk students 

7.7 

6.0 

Y '-j ** 

0.040 

Moderate-risk students 

6.4 

5.3 

i.i * 

0.063 

Individual meeting for support during a life-changing event 




High-risk students 

2.9 

1.6 

1.3 ** 

0.012 

Moderate-risk students 

2.3 

1.9 

0.4 

0.368 

Individual meeting about personal goals and behavior 





High-risk students 

5.3 

4.8 

0.5 

0.509 

Moderate-risk students 

4.6 

3.7 

0.9 

0.136 

Group meeting about academics 





High-risk students 

5.0 

3.6 

1.4 ** 

0.043 

Moderate-risk students 

4.7 

3.6 

Y y ** 

0.039 

Group meeting for support during a life-changing event 





High-risk students 

2.3 

2.1 

0.2 

0.714 

Moderate-risk students 

1.8 

1.4 

0.4 

0.263 

Group meeting about personal goals and behavior 





High-risk students 

3.9 

3.9 

- 0.1 

0.933 

Moderate-risk students 

3.2 

2.9 

0.3 

0.521 

Group meeting for social activities 





High-risk students 

4.7 

2.7 

2.0 *** 

0.004 

Moderate-risk students 

5.3 

2.6 

2 ^ *** 

0.000 

Met with an adult at school to set specific goals for the 

year (%) f 



High-risk students 

71.2 

55.5 

15.7 *** 

0.000 

Moderate-risk students 

Could earn rewards for improving grades, attendance, 
or behavior, or for reaching or making 
progress toward goals (%) f 

65.6 

46.5 

IQ 1 *** 

0.000 

High-risk students 

57.8 

53.4 

4.4 

0.309 

Moderate-risk students 

An adult in school connected student to support 
programs or help outside of school (%) f 

60.4 

53.8 

6.6 ** 

0.048 

High-risk students 

54.7 

42.6 

12.1 *** 

0.005 

Moderate-risk students 

48.3 

38.0 

10.4 *** 

0.002 

Number of students (total = 1,416) 

701 

715 



High-risk students (total = 536) 

277 

259 



Moderate-risk students (total = 880) 

424 

456 
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Appendix Table C.3 (continued) 


SOURCE: MDRC calculations based on the spring 2014 follow-up student survey. 

NOTES: The analyses reported in this table are based on the stable student survey analysis sample, which 
includes all students who responded to the spring 2013 and 2014 follow-up student surveys. 

The estimated differences between the case-managed group and the non-case-managed group are 
regression-adjusted, controlling for random assignment blocks by school as well as the following baseline 
characteristics: race, gender, free or reduced-price lunch status, English as a second language, whether 
qualified for a gifted program, and special education status. 

The values in the column labeled “Case-Managed Group” are the observed means for students 
randomly assigned to the case-managed group. The “Non-Case-Managed Group” values are the 
regression-adjusted means for students randomly assigned to the non-case-managed group, using the 
observed mean covariate values for the case-managed group as the basis for the adjustment. 

High-risk students include those students who were chronically absent, who failed at least one core 
course, or who were ever suspended in the baseline (2011-2012) school year. Moderate-risk students 
include those who were never chronically absent, never failed a core course, and were never suspended in 
the baseline year. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between the case-managed and non-case-managed 
groups. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The sample size reported in the table is for the stable student survey sample. However, the sample size 
varies across outcomes due to missing data. The percentage of missing data for any given outcome does 
not exceed 5 percent. 

A dagger (f) indicates that there is a statistically significant difference between the estimated effects 
on moderate- and high-risk students at the 5 percent level. 

a The times-per-year measure was created by combining a measure of duration (“for how long did you 
do this activity?”) and a measure of frequency (“how often did you do this activity?”) for each of the 
above support activities in the spring 2014 follow-up student survey. The duration measure was converted 
from a four-point scale to an estimate of the number of weeks per school year a student participated in the 
activity: “I never did this activity” = 0 weeks, “less than half of the school year” = 9 weeks, “about half of 
the school year” = 18 weeks, and “most or all of the school year” = 27 weeks. The frequency measure was 
converted from a four-point scale to an estimate of the number of times per week a student participated in 
the given activity: “I never did this activity” = 0, “less than once a month” = 0.125, “1-2 times a month” = 
0.375, and “one or more times a week” = 1. 
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Appendix Table C.4 

Effects on Students’ Attitudes and Activities After One Year 


Case- Non-Case- P-Value for 

Managed Managed Estimated Effect Estimated 
Survey Item_Group_Group Effect Size_Effect 


Caring adult at home (l-4) a 

Caring adult at school (1 -4) b 

Caring adult outside of home or school (l-4) c 

Friend quality (1 -4) d 

School-sponsored extracurricular activities 
Students selecting at least 1 activity (%) 

Mean number of activities done sometimes 
Mean number of activities done often 

Non-school-sponsored extracurricular activities 
Students selecting at least 1 activity (%) 

Mean number of activities done sometimes 
Mean number of activities done often 

Student engagement with school (l-4) e 

Educational attitudes (1-4) 1 
Positive educational self-perception and effort® 
Negative educational self-perception and effort 1 
Positive valuation of education 1 

How far would you like to go in school? (%) 
Some high school 
Finish high school 

Some college or trade/technical school 
Finish college or trade/technical school 
Graduate school after college 
Don’t know 


3.38 

3.31 

0.07 

0.10 ** 

0.021 

3.21 

3.13 

0.08 

o n ** 

0.015 

3.45 

3.35 

0.10 

0.13 *** 

0.008 

2.94 

2.88 

0.06 

0.10 ** 

0.026 


84.0 

80.9 

3.1 

0.08 

0.124 

2.92 

3.0 

-0.13 

-0.03 

0.497 

2.20 

2.3 

-0.12 

-0.04 

0.425 


78.2 

79.8 

-1.6 

-0.04 

0.454 

1.96 

1.9 

0.01 

0.00 

0.924 

1.58 

1.6 

-0.05 

-0.02 

0.612 

2.76 

2.71 

0.05 

0.06 

0.172 

3.00 

2.96 

0.04 

0.08 * 

0.067 

2.66 

2.64 

0.02 

0.03 

0.502 

2.13 

2.09 

0.04 

0.05 

0.233 

3.31 

3.21 

0.10 

0.15 *** 

0.001 





0.339 

1.2 

2.0 

-0.8 

-0.06 


12.5 

9.0 

3.5 

0.12 


5.2 

4.5 

0.7 

0.03 


25.3 

29.3 

-4.0 

-0.09 


45.5 

44.8 

0.7 

0.01 


10.4 

10.8 

-0.4 

-0.01 



(continued) 
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Appendix Table C.4 (continued) 


Survey Item 

Case- 

Managed 

Group 

Non-Case- 

Managed Estimated 
Group Effect 

Effect 

Size 

P-Value for 

Estimated 

Effect 

How far do you think you will actually go 
Some high school 

in school? (%) 

3.2 

2.5 

0.7 

0.04 


Finish high school 

13.8 

14.6 

-0.8 

-0.02 


Some college or trade/technical school 

8.9 

10.1 

-1.2 

-0.04 


Finish college or trade/technical school 

27.6 

28.2 

-0.6 

-0.01 


Graduate school after college 

30.9 

30.1 

0.7 

0.02 


Don’t know 

15.7 

14.8 

0.9 

0.03 


Number of students (total = 1,416) 

701 

715 





SOURCE: MDRC calculations based on the spring 2013 follow-up student survey. 

NOTES: The analyses reported in this table are based on the stable student survey sample, which includes 
all students who responded to the spring 2013 and spring 2014 follow-up student surveys. See Appendix D 
for survey questions. 

Estimated effects are regression-adjusted using ordinary least squares, controlling for random 
assignment blocks by school, as well as the following baseline characteristics: race, gender, free or 
reduced-price lunch status, English as a second language, whether qualified for a gifted program, special 
education status, and a baseline measure of the outcome variable. 

The values in the column labeled “Case-Managed Group” are the observed means for students randomly 
assigned to the case-managed group. The “Non-Case-Managed Group” values are the regression-adjusted 
means for students randomly assigned to the non-case-managed group, using the observed mean covariate 
values for the case-managed group as the basis for the adjustment. 

Effect sizes are calculated by dividing the impact estimate by the standard deviation of the outcome 
measure for students in the stable student survey sample who are in the non-case-managed group. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between case-managed and non-case-managed groups. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The sample sizes reported in the table are for the stable student survey sample. However, the sample 
sizes vary across outcomes due to missing data. The percentage of missing data for any given outcome does 
not exceed 1 percent. 

a Scale based on responses to survey questions 17a-17g, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach’s alpha = 0.89. 

b Scale based on responses to survey questions 8 a- 8 f, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach’s alpha = 0.86. 

c Scale based on responses to survey questions 15a-15f, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach’s alpha = 0.93. 

d Subscale based on responses to survey questions lOa-lOf, ranging from 1 = “not at all true” to 4 = 

“very much true.” Cronbach's alpha = 0.71. 

e Scale based on responses to survey questions 9a-9e, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach’s alpha = 0.84. 

f Scale based on responses to survey questions 1 la-1 In, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach’s alpha = 0.83. 

g Subscale based on responses to survey questions lib, lie, Ilf, 1 lg, ranging from 1 = “not at all true” 
to 4 = “very much true.” Cronbach’s alpha = 0.72. 

h Subscale based on responses to survey questions 11a, 11c, lid, 1 lh, ranging from 1 = “not at all true” 
to 4 = “very much true.” Cronbach's alpha = 0.75. 

‘Subscale based on responses to survey questions 1 lj-1 In, ranging from 1 = “not at all true” to 4 = “very 
much true.” Cronbach's alpha = 0.79. 
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Appendix Table C.5 

Effects on Students’ Academic and Behavioral Outcomes After One Year 


Outcome 

Case- 

Managed 

Group 

Non-Case- 

Managed 

Group 

Estimated 

Effect 

Effect 

Size 

P- Value for 

Estimated 

Effect 

Main outcomes 

Chronic absenteeism 11 (%) 

16.0 

13.9 

2.1 

0.06 

0.277 

Failed at least 1 core course (%) 

29.6 

29.7 

-0.2 

0.00 

0.928 

School progress 

Core credit accumulation for graduation 1 (%) 21.1 

21.7 

-0.6 

-0.07 

0.386 

Behavior 

Average attendance rate (%) 

94.2 

94.5 

-0.3 

-0.04 

0.410 

Number of suspensions 

1.6 

1.5 

0.1 

0.03 

0.469 

Academic achievement 

Average core course marks (%) 

79.6 

79.9 

-0.3 

-0.04 

0.277 

Number of students (total = 1,501) 

751 

750 





SOURCE: MDRC calculations based on student records obtained from school districts. 

NOTES: The analyses reported in this table are based on the stable school records sample, which 
includes all students with course-failure data for the 2012-2013 school year. 

Estimated effects are regression-adjusted using ordinary least squares, controlling for random 
assignment blocks by school, as well as the following baseline characteristics: race, gender, free or 
reduced-price lunch status, English as a second language, whether qualified for a gifted program, special 
education status, and a baseline measure of the outcome variable. 

The values in the column labeled “Case-Managed Group” are the observed means for students 
randomly assigned to the case-managed group. The “Non-Case-Managed Group” values are the 
regression-adjusted means for students randomly assigned to the non-case-managed group, using the 
observed mean covariate values for the case-managed group as the basis for the adjustment. 

Effect sizes are calculated by dividing the impact estimate by the standard deviation of the outcome 
measure for students in the school records analysis sample who are in the non-case-managed group. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between case-managed and non-case-managed groups. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The sample size reported in the table is for the stable school records sample. However, the sample 
size varies across outcomes due to missing data. The percentage of missing data is 24 percent for 
attendance rates and absenteeism, 4 percent for suspensions, 6 percent for credit accumulation, and 0 
percent for other outcomes. 

a A student is considered chronically absent if he or she has an attendance rate below 90 percent. 

b This outcome is for high school students only; the state-level graduation requirements for the four 
core subjects are used in this calculation. A student can be expected to accumulate about 25 percent of 
core credits in each year of high school. 
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Appendix Table C.6 

Effects on Ninth-Grade and Non-Ninth-Grade Students’ 
Academic and Behavioral Outcomes 


Outcome 

Case- 

Managed 

Group 

Non-Case- 

Managed 

Group 

Estimated 

Effect 

Effect 

Size 

P-Value for 

Estimated 

Effect 

Students in ninth grade at random assignment 

Main outcomes 

Chronic absenteeism 3 (%) f 

21.4 

25.4 

-4.0 

- 0.10 

0.487 

Failed at least 1 core course (%) f 

42.7 

41.9 

0.9 

0.02 

0.898 

School progress 

Core credit accumulation for graduation (%) f 

20.3 

18.3 

2.0 * 

0.22 

0.066 

Behavior 

Average attendance rate (%) 

92.1 

90.0 

2.1 

0.21 

0.178 

Number of suspensions 

2.1 

1.2 

0.98 * 

0.33 

0.053 

Academic achievement 

Average core course marks (%) f 

77.6 

76.6 

1.0 

0.13 

0.330 

Sample size (total = 258) 

131 

127 





Students in tenth- and eleventh-grade at random assignment 


Main outcomes 

Chronic absenteeism 3 (%) f 

33.9 

28.2 

5.7 

0.14 

0.362 

Failed at least 1 core course (%) f 

44.2 

29.8 

14 4 ** 

0.31 

0.020 

School progress 

Core credit accumulation for graduation (%) f 

18.7 

22.1 

_3 4 *** 

-0.37 

0.002 

Behavior 

Average attendance rate (%) 

89.9 

89.3 

0.6 

0.06 

0.697 

Number of suspensions 

0.4 

0.3 

0.16 

0.05 

0.261 

Academic achievement 

Average core course marks (%) f 

76.6 

81.0 

_4 4 *** 

-0.56 

0.000 

Sample size (total = 230) 

120 

110 
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Appendix Table C.6 (continued) 


SOURCE: MDRC calculations based on student records obtained from school districts. 

NOTES: The analyses reported in this table are based on the stable school records sample. The estimated 
differences between the case-managed group and the non-case-managed group are regression-adjusted using 
ordinary least squares, controlling for the blocking of random assignment by school and grade level at random 
assignment, as well as the following baseline characteristics: race, gender, free or reduced-price lunch status, 
English as a second language, whether qualified for a gifted program, and special education status. The values 
in the column labeled “Case-Managed Group” are the observed means for students randomly assigned to the 
case-managed group. The “Non-Case-Managed Group” values in the next column are the regression-adjusted 
means for students randomly assigned to the non-case-managed group, using the observed distribution of the 
case-managed group across random assignment blocks as the basis for the adjustment. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Effect sizes are calculated by dividing the impact estimate by the standard deviation of the outcome measure 
for students in the stable school records sample who are in the non-case-managed group. 

A two-tailed t-test was applied to differences between the case-managed and non-case-managed groups. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A dagger (f) indicates that there is a difference between the estimated effects on ninth-grade and non-ninth- 
grade students that is statistically significant at the 5 percent level. 

a A student is considered to be chronically absent if he or she has an attendance rate of less than 90 percent. 
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Appendix Table C.7 

Effects on Sixth-Grade and Non-Sixth-Grade Students’ 
Academic and Behavioral Outcomes 


Outcome 

Case- 

Managed 

Group 

Non-Case- 

Managed 

Group 

Estimated 

Effect 

Effect 

Size 

P-Value for 

Estimated 

Effect 

Students in sixth grade at random assignment 

Main outcomes 

Chronic absenteeism 11 (%) f 

15.7 

17.9 

-2.2 

-0.05 

0.484 

Failed at least 1 core course (%) f 

17.6 

21.5 

-4.0 

-0.09 

0.209 

School progress 

Core credit accumulation for graduation (%) 

NA 

NA 

NA 

NA 

NA 

Behavior 

Average attendance rate (%) 

94.2 

93.9 

0.2 

0.02 

0.708 

Number of suspensions 

1.6 

1.6 

0.08 

0.03 

0.793 

Academic achievement 

Average core course marks (%) 

81.5 

80.8 

0.7 

0.09 

0.143 

Sample size (total = 491) 

256 

235 





Students in seventh and eighth grade at random assignment 


Primary outcomes 

Chronic absenteeism 1 (%) f 

20.6 

18.3 

2.3 

0.06 

0.469 

Failed at least 1 core course (%) f 

36.7 

33.5 

3.2 

0.07 

0.401 

School progress 

Core credit accumulation for graduation (%) 

NA 

NA 

NA 

NA 

NA 

Behavior 

Average attendance rate (%) 

92.7 

93.5 

-0.7 

-0.08 

0.177 

Number of suspensions 

1.7 

1.5 

0.21 

0.07 

0.443 

Academic achievement 

Average core course marks (%) 

78.6 

79.1 

-0.5 

-0.06 

0.422 

Sample size (total = 518) 

240 

278 
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Appendix Table C.7 (continued) 


SOURCE: MDRC calculations based on student records obtained from school districts. 

NOTES: The analyses reported in this table are based on the stable school records sample. The estimated 
differences between the case-managed group and the non-case-managed group are regression-adjusted using 
ordinary least squares, controlling for the blocking of random assignment by school and grade level at random 
assignment, as well as the following baseline characteristics: race, gender, free or reduced-price lunch, 

English as a second language, whether qualified for a gifted program, and special education status. The values 
in the column labeled “Case-Managed Group” are the observed means for students randomly assigned to the 
case managed group. The “Non-Case-Managed Group” values in the next column are the regression-adjusted 
means for students randomly assigned to the non-case managed group, using the observed distribution of the 
case-managed group across random assignment blocks as the basis for the adjustment. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Effect sizes are calculated by dividing the impact estimate by the standard deviation of the outcome 
measure for students in the stable school records sample who are in the non-case-managed group. 

A two-tailed t-test was applied to differences between the case-managed and non-case-managed groups. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

NA = not applicable. 

A dagger (f) indicates that there is a difference between the estimated effects on sixth-grade and non-sixth- 
grade students that is statistically significant at the 5 percent level. 

a A student is considered to be chronically absent if he or she has an attendance rate of less than 90 percent. 
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